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FOREWORD 


This  report  summarizes  work  to  date  on  developing  a  method  for 
deriving  job  standards  from  system  effectiveness  criteria.  It  is 
composed  of  two  volumes.  Volume  I  describes  the  technical  work  in 
developing  the  standards  derivation  method.  Volume  H,  which  is 
classified  confidential,  reports  application  of  the  method  to  the  AN/SPS-40 
radar. ^ 

The  study  was  initiated  under  the  auspices  of  Captain  R.  M.  Stuart, 
and  lias  been  continued  under  Captain  L.  A.  Wilder,  USN,  Director, 
Personnel  Research  Division,  Bureau  of  Naval  Personnel.  Mr.  Sidney 
Friedman,  Head,  Psychological  Research  Branch  (Pers-152)  is 
Scientific  Officer  and  Contract  Monitor  is  Dr.  Martin  Wiskoff.  Generous 
assistance  in  data  collection  was  provided  by  Mr.  A.  A.  Sjoholm  and 
Mr.  W.  Hopkins  (Pers-153)  and  Dr.  J.  R.  Curtin  and  Mr.  S.  J.  Sokol 
(BuShips  742C).  The  project  staff  is  grateful  to  RADM  E.  C.  Ruckner, 
ACNO  (Training)  and  Mr.  J.  J.  Collins  (OP~07T)  for  their  support  and 
interest  in  the  study. 

Responsible  officer  for  Dunlap  and  Associates,  Inc.,  is  Dr.  Joseph 
W.  Wulfeck,  Dr.  Robert  E.  Blanchard  has  served  as  Project  Director. 
Staff  members  for  the  study  have  been  Mr.  R.  A.  Westland,  Mr.  M.  B. 
Mitchell,  Dr.  A.  J.  Hoisman,  Mr,  C.  A.  Brictson  and  Mr.  A.  M.  Daitch. 

Several  Navy  facilities  were  visited  while  selecting  a  test  subsystem 
for  use  in  method  development  and  application  and  later  during  data 
collection  and  procedural  verification  on  the  AN/SPS-40  radar.  The 
cooperation  of  those  facilities  was  outstanding  and  their  contribution  to  the 
progress  of  the  study  is  gratefully  acknowledged: 

.  Commander  Training  Pacific,  San  Diego 

.  Fleet  Anti-Warfare  Training  Center,  San  Diego 
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SUMMARY 


The  cost  and  complexity  of  modern  Naval  systems  and  the  increasing 
difficulty  in  achieving  optimal  use  of  Naval  personnel  and  training  resources 
establish  the  need  for  a  method  for  determining  objectively  the  level  of 
personnel  performance  necessary  to  meet  Naval  system  operational 
req^zirements .  The  purpose  of  this  study  was  to  develop  such  a  method 
with  a  basic  capability  for: 

,  Deriving  specific  personnel  performance  standards 
with  definable  relations  to  ultimate  system 
effectiveness  requirements;  and 

.  Determining  the  effect  on  system  effectiveness  of 
personnel  performance  levels  that  deviate  from 
established  performance  standards. 

To  assist  in  formulating  an  approach  to  method  development,  existing 
methods  and  techniques  for  relating  personnel  performance  to  system 
effectiveness  were  appraised  for  relevance  to  the  present  study  and  were 
modified  and  extended  as  appropriate.  Also  to  aid  in  approach  forniudation, 
a  conceptual  framework  was  developed  that  coniprises  a  derivative  and 
an  integrative  process.  The  derivative  process  involves  determination 
of  the  personnel  performance  requirements  (job  standards)  imposed  by  a 
particular  system  design  on  associated  personnel/ equipment  functional 
units;  whereas  the  integrative  process  involves  determination  Of  the 
degree  to  which  a  set  of  available  personnel  capabilities  can  fulfill  stated 
system  requirements  and  the  effect  on  system  operational  goals  of 
performance  levels  that  deviate  from  established  performance  standards. 

The  method  developed  comprises  a  set  of  procedures  for  using  analytic 
and  probabilistic  tools  which  organize  system  effectiveness  requirements 
and  other  types  of  system  descriptive  data  to  enable  determination  of 
system-related  personnel  performance  standards  at  a  given  level  of 
specificity.  The  types  of  input  data  required  by  the  method  in  its  present 
form  are;  (1)  operational  requirements  data,  (2)  system  descriptive 
data,  and  (3)  human  capability  data. 

As  conceived  in  the  study,  a  job  standard  ultimately  will  be  composed 
of  a  personnel/equipment  functional  unit,  an  accuracy/time  requirement 
and  a  required  probability  of  successful  performance.  In  the  study  to  date, 
effort  has  been  concentrated  on  describing  and  organizing  personnel/ 
equipment  functional  xmits  and  in  treating  the  probability-of-successful- 
performance  component  of  the  job  standard.  However,  preliminary  work 
has  been  done  to  explore  teclmiques  for  incorporating  performance  time 
and  accuracy  as  job  standard  components. 
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To  employ  the  method,  system  data  are  used  to  const:ruct  graphic 
and  mathematical  models  of  system  operation.  The  Graphic  State 
Sequence  Model  (GSSM)  is  a  diagrammatic  representation  of  the  system 
which  identifies  the  various  means  by  which  system  effectiveness  require¬ 
ments  may  be  satisfied.  The  GSSM  provides  for  determination  of 
sequential  relations  among  personnel/equipment  functions  by  specifying 
the  state  of  the  system  prior  to  and  following  each  function.  The  graphic 
model  allows  for  establishment  of  the  personnel/equipment  functional 
units  (PEF  Units)  and  supports  determination  of  accuracy  and  time 
components  of  the  perfoi'mance  standard. 

The  Mathematical  State  Sequence  Model  (MSSM)  utilizes  conventional 
probability  theory  to  describe,  in  equation  form,  the  mathematical 
relations  among  the  units  composing  the  GSSM.  Since  effectiveness 
requirements  are  generally  established  for  overall  system  operations  or 
major  system  functions,  standards  derivation  becomes  a  process  of 
allocating  those  effectiveness  requirements  among  personnel/equipment 
functional  units  (.he  derivative  process).  An  interim  allocation  technique 
was  developed  uhxch  employs  pooled,  experimentally-derived  data  on 
human  reliability  to  (1)  determine  the  relation  among  the  personnel/ 
equipment  functional  units  composing  the  system  and  (2)  to  establish  the 
probability  component  of  the  job  standard  through  the  mathematical 
process. 

To  extend  the  basic  capability  of  the  method,  two  other  problem 
dimensions  were  considered:  (1)  personnel  performance  standards  for 
maintenance  activities,  and  (2)  additional  job  standard  components  (time 
and  accuracy).  It  wes  determined  that  maintenance  actions  can  be 
modeled  using  the  graphic  and  mathematical  state  sequence  modeling 
techniques  of  the  basic  standards  method.  Overall  maintenance  time 
requirements  can  be  derived  readily  from  system  availability  require¬ 
ments  if  system  failure  rate  input  data  are  available.  The  performaxice 
time  component  of  the  job  standard  can  then  be  determined  for  various 
categories  of  system  maintenance. 

Some  hypotheses  v/ere  developed  regarding  the  relations  between 
probability  of  accomplishment  and  the  two  additional  job  standard 
components,  time  and  accuracy.  Those  relations  are  presented  as  a  set 
of  equations  interrelating  an  Index  of  Task  Accomplishment  (IOTA)  and 
the  probability,  time  and  accuracy  components  of  the  standard.  While 
application  of  the  equations  is  straightforward,  it  was  empliasized  that 
their  use  depends  upon  (1)  experimental  determination  of  certain  unknown 
human  capability  values  and  (2)  verification  of  their  validity  by  applying 
them  to  data  gathered  under  controlled,  observational  conditions. 
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The  method  was  applied  to  data  obtained  on  the  AN/SPS>4C  radar 
subsystem  to  test  and  refine  various  elements  of  the  procedures.  It 
was  determined  that  the  method  can  be  applied  successfully  to  a  suo- 
system  such  as  the  AN/SPS-40  radar  and  that  is  likely  will  be  applicable 
to  any  Navy  system  for  which  the  necessary  input  data  are  available. 

During  the  test  application^  a  procedural  format  was  developed  to 
facilitate  use  of  the  method  by  skilled  systems  analysts. 

Due  to  the  capability  of  the  job  standard  method  (1)  to  derive  system- 
related  performance  standards  and  (2)  to  test  the  effects  of  performance 
levels  that  deviate  from  established  standards,  the  method  has  broad 
potential  application  in  personnel  research  and  as  a  general  research 
tool  in  system  design  and  development.  Use  of  job  standards  as  appraisal 
criteria  that  reflect  actual  system  operational  requirements  in  the  areas 
of  selection  and  training  seems  quite  promising.  The  method  also  appears 
useful  as  an  aid  in  developing  occupational  hierarchies  for  personnel 
subsystems.  A  preliminary  investigation  of  the  ueefulness  of  the  method 
for  that  purpose  was  performed  in  the  present  study.  As  a  system  design 
and  development  tool,  the  method  has  high  potential  value  in  trade-off 
analysis  and  in  testing  the  compatibility  of  various  system  design  concepts 
relative  to  specified  operational  requirements. 

To  extend  and  refine  the  basic  capability  of  the  job  standards  derivation 
method  reported  here,  additional  methodological  research,  emf  "rical 
testing  and  consideration  of  practical  problems  of  implementation  and 
application  is  recommended. 
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I.  INTRODUCTION 


A.  Purpose 

The  complexity  of  modern  Naval  systems  has  made  it  increasingly 
difficult  to  determine  the  levels  of  performance  system  personnel  must 
meet  in  order  that  the  system  as  a  whole  can  meet  its  assigned  effective¬ 
ness  goals.  If  personnel  performance  levels  are  set  too  low,  system 
effectiveness  is  likely  to  be  unacceptable;  if  they  are  set  too  high,  the 
Navy  spends  more  than  necessary  to  select  and  train  personnel  to  man 
the  system.  In  the  second  case,  at  least,  system  performance  is  not 
degraded,  although  that  is  small  consolation  since  the  Navy  does  not 
have  infinite  resources  and  cannot  afford  overdesign. 

The  Navy  has  expended  a  great  deal  of  effort  to  develop  and  improve 
techniques  that  reveal  the  what,  wlien  and  why  of  personnel  performance 
in  Naval  systems;  however,  little  has  been  done  to  find  a  way  of  objectively 
relating  quantitative  levels  of  personnel  performance  to  ultimate  system 
effectiveness  requirements.  For  the  most  part,  subject  matter  experts 
make  estimates  of  required  personnel  performance  levels.  In  some  cases, 
performance  measures  are  obtained  from  a  specific  group  oi  personnel 
who  are  presumed  to  have  the  skills  and  abilities  that  the  system  is 
thought  to  require,  and  the  results  are  used  to  specify  desirable  per¬ 
sonnel  performance  levels.  That  approach  overlooks  objective  deter¬ 
mination  of  the  personnel  performance  the  system  requires  and  rests 
entirely  on  what  knowledgeable  people  think  it  requires.  Unfortunately, 
today's  complex  Naval  systems  cannot  easily  tolerate  the  risk  inherent  in 
the  use  of  subjective  criteria.  Performance  levels  set  in  that  manner  may 
be  too  high,  too  low,  or  right  on  the  button,  but  who  is  to  know  which? 

To  remedy  that  situation,  a  set  of  personnel  goals  or  minimum  job 
standards  with  definable  relations  to  ultimate  system  effectiveness 
requirements  is  needed  to  provide  a  systematic  basis  for  relating  system 
performance  requirements  to  levels  of  personnel  performance.  Such 
performance  standards  would  provide  objective,  system-related  criterion 
data  to  persons  responsible  for  selecting,  training  and  evaluating 
personnel. 

In  addition,  a  means  should  be  available  to  estimate  the  extent  to  which 
system  effectiveness  would  be  affected  by  deviations  from  established 
performance  standards.  If  it  is  determined  that  a  group  being  considered 
for  assignnient  to  a  system  cannot  meet  a  number  of  performance  standards, 
it  is  important  to  be  able  to  determine  the  specific  effect  of  that  particular 


personnel  group's  ‘ubslandard  performance  on  system  operation.  The 
results  of  such  a  test  would  provide  an  indication  of  the  amount  of  degrada¬ 
tion  in  system  effectiveness  that  could  be  expected  from  assigning  that 
group  to  the  system. 

Therefore,  the  purpose  of  this  stody  centers  around  developing  a 
methodology  that  would  fulfill  two  objectives: 

.  Derivation  of  specific  personnel  performance  standards 
with  definable  relations  to  ultimate  system  effectiveness 
requirements;  and 

.  Determination  of  the  effect  on  system  effectiveness  of 
performance  lev'  Is  that  deviate  from  established 
performance  standards. 


B.  Scope 

Most  complex  method  development  studies  can  be  viewed  as  continuous 
efforts  with  some  reasonably  well-defined  milestones.  The  study  effort 
reported  here  was  devoted  to  developing  a  basic  capability  for  meeting  the 
two  objectives  stated  previously.  Our  primary  milestones  were: 

.  Development  of  basic  tools, 

.  Test  applications  of  those  tools, 

.  Refinement  of  the  method  based  on  findings  resulting 
from  application,  and 

.  Consideration  of  additional  problem  dimensions  to  broaden 
the  method's  utility. 

Generally,  the  effort  has  consisted  of  reviewing  existing  methods  and 
tec.  aques  for  relating  personnel  performance  to  system  effectiveness 
and  combining,  modifying  and  extending  those  methods  and  techniquer  as 
required  to  quantify  the  relation  between  system  processes  and  criteria  of 
system  ef .ectiveness. 

To  insure  that  the  method  would  have  practical  utility  and  to  test  its 
applicability  empirically,  it  has  been  applied  to  the  AN/SPS-40  radar 
which  is  an  installed  shipboard  subsystem  (see  Volume  11  of  the  report). 
That  application  demonstrated  the  usefulness  of  the  method  as  well  as 
its  present  limitations.  During  that  test  application,  a  procedural  format 
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was  dftv’^loped  for  apply^ing  the  method  (see  Appendix  B).  The  procedural 
format  permits  use  of  the  method  by  skilled  systems  analysts  and  does 
not  require  the  use  of  advanced  mathematics. 

C.  Products 


Products  of  the  study  are: 

1.  A  critique  of  the  literature  dealing  with  methods  and  techniques 
for  quantifying  the  relation  between  personnel  performance  and 
ultimate  system  effectiveness  requirements. 

2.  A  method  or  series  of  procedures  that  provide  for: 

a.  Identifying  personnel/equipment  functional  units  com¬ 
posing  a  system  or  subsystem  and  defining  their 
functional  relation  to  required  outputs. 

b.  Stating  the  contribution  of  each  personnel/equipment 
functional  unit  to  system  operation, 

c.  Establishing  performance  standards  for  each  per¬ 
sonnel/equipment  functional  unit  that  can  be  related 
directly  to  ultimate  system  effectiveness  requirements. 

d.  Testing  the  effects  on  system  effectiveness  of  per¬ 
formance  levels  that  deviate  from  the  established 
performance  standards. 

3.  An  empirical  test  of  the  adequacy  and  practicality  of  the  method 
using  data  obtained  on  an  operational  Navy  subsystem  (AN/SPS-40). 

4.  A  body  of  performance  standards  for  the  personnel/equipment 
functional  units  identified  for  the  AN/SPS-40  test  subsystem. 

5.  Preliminary  investigation  of  the  usefulness  of  the  method  as  an 
aid  in  constructing  occupational  hierarchies  (position  definitions) 
using  data  resulting  from  the  AN/SPS-40  analysis. 

6.  A  procedural  format  that  will  permit  use  of  the  method  by  the 
typical  sy'  -ms  analyst  and  which  will  not  require  skills  in 
advanced  r.-Athematics . 
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II.  APPROACH  FORMULATION 


FormiiJating  an  approach  for  developing  a  method  for  establishing 
system-related  job  standards  involved  tv/o  steps: 

.  Detailed  examination  of  various  existing  techniques  for 
identifying  personnel/equipment  system  processes  and 
for  quantifying  the  relation  of  those  processes  with 
criteria  of  system  effectiveness. 

Development  of  a  conceptual  framework  for  use  as  a 
specific  guide  for  method  development  and  to  maintain 
continuity  of  purpose  with  study  objectives. 

A  brief  overview  of  related  research  and  a  detailed  description  of 
the  conceptual  framework  for  the  approach  are  given  below. 

A.  Overview  of  Related  Research 


In  order  to  assay  methods  currently  used  to  evaluate  personnel  per¬ 
formance  and  general  tools  of  systems  analysis,  a  detailed  literature 
review  was  conducted  focusing  on  existing  system  analysis  tools  and  their 
application  to  military  system  analysis  and  evaluation.  Over  three 
hundred  articles  and  reports  have  been  reviewed.  A  detailed  critique  of 
the  major  studies  has  been  submitted  as  a  separate  project  product.  1 
Principal  findings  and  brief  summaries  of  the  more  important  literature 
investigated  are  reported  here. 

For  convenience  of  reporting,  system  techniques  are  divided  into  two 
classes:  techniques  that  are  essentially  conceptual  or  graphic,  and  those 
that  are  mathematical.  Because  of  the  special  need  of  the  study  for 
quantifiable  system  measures,  current  emphasis  in  reviewing  the  literature 
is  in  the  area  of  mathematical  modeling  techniques.  That  emphasis  is 
reflected  in  both  the  number  and  kind  of  report  reviewed. 

A  list  of  selected  references  including  the  citations  referenced  in 
this  summary  of  the  literature  review,  as  well  as  other  reports  germane 
to  the  job  standards  subject  area,  is  provided  in  Appendix  A. 


Hoisman,  A.J.,  and  Daitch,  A.  M.  Techniques  for  Relating  Personnel 
Performance  to  Systems  Effectiveness  Criteria:  A  Critical  Review  of 
the  Literature,  Santa  Monica,  California:  Dunlap  and  Associates,  Inc., 
September  1964. 
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1 .  Non-Mathematical  Techniques 


While  the  literature  of  system  analysis  and  evaluation  has  a  long 
and  varied  history  (cf.  Whitehead,  1926;  Chapman,  1952;  Hoag,  1956),  it 
was  not  imtil  the  late  1950' s  that  major  effort  was  directed  toward  develop¬ 
ing  techniques  of  system  analysis  that  were  general  rather  than  system 
specific.  The  most  tangible  evidence  of  heightened  interest  at  that  time 
may  be  seen  in  the  work  sponsored  by  the  Office  of  Naval  Research  to 
synthesize  systems  research  methodology  (McGrath,  Nordlie  and  Vaughn, 
I960;  McGrath  and  Nordlie,  1959;  McGrath  and  Nordlie,  I960;  and 
Havron,  1961).  Although  that  ambitious  program  has  identified  many 
problem  areas  in  the  systems  analysis  field,  no  solutions  that  would  have 
direct  relevance  to  job  standards  methodology  have  been  proposed. 

Examples  of  specific  methods  of  system  analysis  are  the  work 
of  Bates,  Schaeffer  and  Shapero  (Shapero  and  Bates,  1959;  Scliaeffer  and 
Shapero,  1961;  and  Schaeffer,  1962).  The  intent  of  their  conceptualization 
is  development  of  methods  of  analysis  which  will  integrate  human  system 
elements  with  an  overall  system  analysis.  The  result  is  an  analytic 
instrument  based  on  the  Systems  Analysis  and  Integration  Model  (SAIM) 
which  consists  of  a  comprehensive  classification  scheme  of  all  major 
system  elements  presented  in  a  descriptive  matrix  format.  The  matrix 
classifies  the  elements  of  a  system  into  three  groups:  those  determining 
the  na.ture  and  form  of  a  system;  those  comprising  the  parts  of  the  system; 
and  those  integrating  the  parts  of  a  system.  Elements  of  the  system  are 
shown  in  a  two-dimensional  square  matrix  which  lists  system  elements  as 
row  and  column  headings.  Entries  in  the  cells  formed  are  used  to  indicate 
direct  connections  between  pairs  of  system  elements.  The  matrix  may 
be  re-entered  as  frequently  as  necessary  to  trace  all  meaningful  (and 
practical)  element  interactions.  The  matrix  produces  a  lattice  of 
connections  representing  the  total  relation  among  the  specified  elements. 

It  is  felt  that  while  the  SAIM  m»«.y  be  helpful  in  some  circumstances 
as  a  tool  to  isolate  system  goals  most  directly  affected  by  human  components 
(in  a  given  system),  its  potential  in  system  analysis  seems  limited  to  use 
principally  as  a  descriptive  technique.  As  such,  there  would  appear  to 
be  no  special  advantage  in  its  use  over,  say,  a  more  diagrammatic 
technique. 

2.  Mathematical  Models  and  Analytic  Techniques 

The  systems  analysis  studies  reviewed  and  summarized  below  may 
be  categorized  as  dealing  with  performance,  availability,  or  allocation  of 
system  requirements.  The  "performance"  category  may  be  classified  as 
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either  human  or  equipment  performance.  Some  of  the  studies  that  treat 
equipment  performance  could  be  adapted  to  treat  human  performance: 
they  will  be  noted  when  appropriate. 

a.  Performance.  The  majority  of  the  systems  studies  reviewed 
here  dealt  with  analysis  of  performance  and  derivation  of  various  per¬ 
formance  measures.  Siegel  and  Wolf  (1961),  Swain  (1963),  and  Young 
(1962)  were  concerned  with  human  performance.  Siegel  and  Wolf  developed 
a  digital  simulation  model  to  evaluate  man's  performance  in  one -operator, 
man-machine  systems.  In  their  model,  operator  performance  in  doing  a 
specific  set  of  subtasks  was  simulated.  They  considered  the  effects  cf 
operator  stress  on  speed  and  accuracy  of  subtask  performance.  Swain's 
work  described  a  human  factors  analysis  which  derived  a  mathematical 
model  of  a  man-machine  system  degradation  resulting  from  human  errors. 
His  model  allowed  for  computation  of  system  or  subsystem  failures  based 
on  specified  human  errors.  A  novel  approach  to  human  factors  analysis 
was  discussed  by  Yoimg  who  treated  the  problem  of  optimizing  scheduling 
of  crew  members  to  tasks.  He  applied  the  classical  assignment  problem 
of  linear  programming  to  assign  men  to  jobs  on  the  basis  of  each 
individual's  task  proficiency. 

The  work  of  Hamilton  et.  al. ,  (I960)  bordered  between  human 
performance  and  system  performance  since  they  developed  a  relation 
between  human  operator  variables  and  system  performance  variables.  The 
system  performance  measure  was  the  probability  of  the  system  success¬ 
fully  completing  its  mission  (some  may  argue  that  this  is  an  effectiveness 
criterion  and  not  a  performance  measure,  but  in  the  context  of  Hamilton's 
report,  it  is  a  performance  measure)  and  it  was  derived  as  a  function  of 
time  remaining  to  complete  the  different  tasks  (detection,  identification, 
etc.)  that  the  operator  performed. 

Studies  that  dealt  with  equipment  performance  without  regard 
a  man-machine  system  context  were  reported  by  Engel  (1963),  White  et.al.  , 
(1963),  and  Magnavox  Research  Laboratories  (1961).  Engel  derived  a 
mathematical  model  that  could  be  utilized  for  estimating  the  tracking 
effectiveness  of  a  surveillance  force  in  continuous  detection  and  tracking 
of  hostile  targets  entering  a  surveillance  region.  His  measure  of  per¬ 
formance  was  the  proportion  of  targets  that  could  be  expected  to  be  under 
surveillance  or  to  be  tracked  at  any  instant  of  time.  Although  Engel  did 
not  mention  human  behavior  specifically,  the  model  could  probably  be 
adapted  to  human  effectiveness  for  a  manual  surveillance /tracking  mode. 

The  work  of  White  et. al.,  was  concerned  with  computing  equipment  per¬ 
formance,  using  Figure -of-Merit  (a  performance  assessment  in  terms  of 
the  user's  requirements)  as  the  measure  of  interest.  They  also  did  not 
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treat  human  behavior,  but  their  techniques  could  be  modified  to  include 
it.  The  Magnavox  study  treated  development  of  a  mathematical  model 
that  could  be  utilized  in  design  of  information  storage  and  retrieval 
systems.  It  derived  the  efficiency  of  a  given  component  in  performing 
a  f  iven  operation  and  an  efficiency  matrix  for  a  system  composed  of  a 
sp.;cific  set  of  components.  Since  a  human  can  be  regarded  as  a  system 
component,  the  model  could  be  adapted  to  include  human  behavior, 

b.  Availability.  The  majority  of  studies  of  availability 
measures  fail  to  treat  human  performance  aspects  in  quantitative  detail. 

A.  study  by  Westland  and  Kanifan  (196?-)  provides  a  compilation  of 
availability  models  applicable  to  a  variety  of  Navy  missions.  Krull  (1963) 
developed  analytical  techniques  that  treated  the  problem  of  a  system 
subject  to  failure  and  subsequent  maintenance  in  which  the  failure  rates 
and  repair  rates  were  considered  to  be  stochastic  variables.  The  system 
was  composed  of  onerational  units  and  repair  channels  with  time -dependent 
failure  rates  and  .pair  rates,  respectively.  The  model  expressed  the 
prchability  that  a  certain  number  of  the  operational  units  would  be  in 
commission  at  any  time,  and  dealt  wdth  reliability  and  maintainability. 

Since  repair  activities  of  maintenance  crews  are  human  functions,  the 
model  can  be  regarded  as  including  human  behavior. 

c.  Allocation  of  System  dequirements.  The  studies  by 
Hamilton  et. al. ,  and  Krull  have  utility  for  allocating  system  require¬ 
ments  to  human  performance  standards.  Hamilton  et.al.  ,  developed  a 
functional  relation  between  the  probability  of  a  system  successfully 
completing  its  mission  and  the  time  necessary  tn  perform  different 
system  functions,  if  the  probability  of  success  is  specified  as  a  system 
requirement,  the  times  to  perform  tasks  or  functions  can  be  allocated. 
Those  times  could  then  be  considered  one  type  of  job  standard. 

Krull*s  model  expressed  the  probability  that  a  certain  number 
of  system  operational  units  would  be  in  commission  at  any  time,  t.  Such 
a  measure  can  be  regarded  as  an  availability  measure  and  it  would  be 
possible  to  use  the  model  to  express  a  maintenance  time  limitation  on 
repair  crews.  For  example,  suppose  that  10  units  out  of  ?  total  of  12 
are  operational  at  time,  tj,  and  it  is  necessary  that  12  units  be  operational 
at  ti;  the  difference  establishes  a  job  standards  requirement  for  the 
maintenance  ciews. 

3.  Remarks 


No  modeling  and  analytical  teci  niques  were  found  to  be  fully 
applicable  in  meeting  the  two  methodological  objectives  of  the  present 
study.  However,  the  concepts  and  techniques  formulated  by  many  of  the 
researchers  aided  materially  in  the  development  of  specific  portions  of 
the  method  reported  here. 
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B.  Conceptual  Framework 


The  key  to  our  conceptual  framework  is  system  mission  and  associated 
system  effectiveness  requirements.  Several  definitions  of  a  "system" 
have  been  offered  m  the  literature;  however,  they  are  generally  inadequate 
for  our  purpose.*-.  It  has  been  said  that  one  man's  system  is  another  man's 
subsystem,  and  so  forth.  Therefore,  after  consideration  of  project  scope 
and  needs,  classes  of  definitions  and  current  Navy  concepts,  it  v/as  decided 
to  define  a  system  ope  rati  ally  as  being  at  the  level  characterized  by 
ASW,  AAW,  Attack  or  Logistics  organizations.  Any  organization  comprised 
of  working  units  at  the  AAW,  ASW,  Attack  and  Logistic  level  would  be 
considered  larger  than  a  system  ...  a  "supra-system.  "  Similarly,  a 
major  functional  component  of  the  level  connoted  by  AAW,  for  example, 
would  be  considered  a  subsystem,  such  as  a  fire  control  radar  subsystem 
oi  a  SAM  missile  subsys'em  of  an  AAW  system. 

A  derivative  and  an  integrative  process  constitute  the  basis  of  the 
conceptual  framework.  The  derivative  process  involves  determining 
the  performance  requirements  imposed  by  a  particular  design  on 
associated  interacting  personnel  and  equipment  /unctions;  whereas  the 
integrative  process  involves  determing  the  degree  to  which  a  set  of 
available  personnel  capabilities  can  fulfill  those  stated  system  requirements. 

1.  The  Derivative  Process 


The  mission  a  system  is  to  fulfill  gives  rise  to  a  body  of  system 
functions  and  a  related  set  oi  system  effectiveness  requirements  which 
define  the  necessary  performance  levels  for  those  functions.  We  have 
selected  the  term  "^ersonnel/Equipment  Functional  Unit  (pEF  Unit)  to 
describe  a  given  man-machine  functional  interaction  at  any  given  level  of 
specificity.  The  relation  of  a  series  of  PEF  Units,  each  of  which  includes 
a  personnel  and  an  equipment  component  (PiEi),  to  a  system  effectiveness 
requirement  (3r)  can  be  expressed  mathematically  as: 

S  =  (P  E  )(P  E  )  ...  (P.E.)  ...(PE), 
r  lizZ  11  nn 

where  n  represents  the  total  number  of  PEF  Units  in  the  system. 

Beginning  with  an  overall  subsystem  function  or  gross  PEF  Unit 
(e.  g. ,  detect  airborne  targets)  and  an  associated  performance  requirement 
(Sr  =  target  detection  at  100  miles  with  probability  .  95),  the  derivative 
process  simultaneously  fractionates  both  the  requirement  (Sj.)  and  the 
function  (FEF  Unit)  from  statements  of  gross  relations  to  statements  of 
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sufficient  behavioral  specificity  that  the  Sr/PEF  Unit  relation  can  be 
employed  effectively  in  setting  the  standards  of  performance.  At  that 
level  of  specificity,  an  Sy  associated  with  its  related  FEF  Unit  represents 
the  performance  standard. 

The  process  of  fractionating  produces  a  network  or  series  of 
traces  which  relate  the  specific  Sr/PEF  Unit  statements  to  overall  system 
effectiveness  requirements.  It  is  important  to  protect  and  sustain  that 
network  throughout  the  derivative  process  since  the  traces  make  it 
possible  (1)  to  derive  job  standards  that  are  related  to  system  effectiveness 
requirements,  and  (2)  to  reflect  variations  in  meeting  job  standards 
quantitatively  in  terms  of  overall  system  effectiveness.  The  pvirpose  of 
the  derivative  process,  then,  is  to  define  specific  functional  units  with 
associated  performance  standards  in  a  form  usable  by  personnel  subsystem 
designers  and  to  establish  a  requirements /PEF  Unit  network  as  a  basis 
for  integrative  processing. 

2.  The  Integrative  Process 


The  concept  of  an  integrative  process  represents  an  attempt  to 
explore  fully  the  potential  usefulness  of  the  method  in  providing  a  means 
for  testing  the  effect  of  a  given  system  state,  expressed  in  terms  of  a 
particular  set  of  personnel  capabilities,  on  specified  system  requirements. 
The  integrative  process  can  be  described  as; 

(p'e')(p'eV..  (p!e!)...  (p'e*)  =  S  . 

1122  11  nn  c 

The  term  Sc  represents  an  estimate  of  system  output  resulting  from  per¬ 
formance  of  the  related  PEF  Units.  The  personnel/equipment  components 
of  the  PEF  Units  above  are  designated  P’  and  E*  to  distinguish  them  from 
the  derivative  case,  although  they  represent  identical  functional  units. 

The  source  of  personnel  capability  data  will  depend  upon  the  type 
of  system,  its  state  of  development  and  the  availability  of  personnel.  If 
a  personnel  group  exists  and  is  being  considered  for  assignment  to  the 
system  (or  must  be  assigned),  capability  data  can  be  obtained  through 
direct  performance  measurement.  If  there  is  no  identifiable  personnel 
group  to  permit  collection  of  capability  data,  then  it  may  be  necessary  to 
obtain  the  data  through  observation  of  proximate  PEF  Units  on  other 
systems,  by  simulation  or  from  collections  of  behavioral  data  available 
in  ’’data  stores. " 

The  integrative  process  involves  assessment  of  PEF  Units  in 
terms  of  human  capabilities.  The  process  originates  at  the  most  specific 
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PEF  Unit/Sc  level  and  proceeds  upward  by  integrating  more  inclusive 
PEF  Unit/Sc  statements  until  an  Sc  i*?  obtained  that  is  general  enough  to 
be  compared  meaningfully  with  an  Sr.  At  that  point,  the  adequacy  of  a 
given  system  design  can  be  appraised  in  terms  of  the  difference  between 
the  Sr  and  the  obtained  Sc- 

In  application,  the  integrative  process  would  be  used  to  test  or 
to  predict  the  contribution  of  a  defined  or  postulated  personnel  gioup's 
capabilities  in  fulfilling  system  effectiveness  requirements.  That 
application  would  be  useful  in  evaluating  what  one  has,  in  terms  of 
personnel  capability  resources,  against  what  a  given  system  configuration 
demands.  The  network  of  PEF  Units  then,  has  a  transitive  or  feedback 
characteristic.  It  must  support  both  derivation  of  specific  Sr/PEF  Unit 
relations  and  integration  of  PEF  Unit/Sc  relations. 

3.  Definition  of  a  Job  Standard 


As  conceived  in  this  study, 
of  the  following  components: 

Description  of  a  personnel/ 
equipment  functional  unit 
(PEF  Unit)  at  a  desired  level 
cf  specificity  including  the 
action  taken  and  the  object  of 
that  action. 

.  Required  accuracy 

Required  performance 
time  to  complete  PEF 
Unit 

.  Required  probability  of 
successfully  completing 
the  action 


job  standards  would  consist  ideally 

Example 

Amplifier  balance  adjusted 
to  zero. 


A  =  ±  5  mv 
T  =  0.5  minutes 

P  =  0.  997 


The  accuracy,  time  and  probability  components  constitute  the  job  standard, 
per  se,  and  represent  fractionated  system  operational  requirements. 


When  combined  for  a  given  PEF  Unit, 
the  following  relations: 


the  components  would  have 
Example 


Time  versus  accuracy  and/or  P  =  .  997  for  adjusting  to 
probability,  if  applicable  ±5  mv  within  0.  3  minutes 

and  to  ±  3  mv  within  0.  5 
minutes 
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Accuracy  versus  probability,  P  of  attaining  ±  5mv=:  .  997 
if  applicable  P  of  attaining  ±  lOmv  =  0.  998 

P  of  attaining  ±  15mv  =  0.  999 


Within  the  context  of  developing  a  basic  capability  in  the  present 
study,  emphasis  was  placed  on  treating  the  probability  component  and  the 
PEF  Unit.  A  discussion  of  the  integration  of  time  and  accuracy  components 
is  provided  in  Section  IV-B. 

4.  Input  Data  Required 

It  was  determined  that  several  types  of  input  data  would  be  required 
within  our  conceptual  framework.  Those  data  types  are: 

a.  Operational  Requirements  Data 


.  Overall  system  effectiveness  requirements 

.  Performance  requirements  for  major  functions 

.  Availability,  reliability  and  maintainability 

requirements 

b.  System  Descriptive  Data 
.  System  design  data 

.  Development  and  test  reports 

Operating  and  maintenance  procedures 

c.  Human  Capability  Data 


.  Performance  times 

.  Probability  of  correct  performance 

,  Variability  of  performance 

The  availability  of  those  various  types  of  system  data  and  the 
degree  to  which  they  can  be  specified  will  vary  among  systems  according 
to  the  nature  and  status  of  the  development  effort.  Operational  require¬ 
ments  data  are  the  most  critical  since  they  are  the  ultimate  basis  for 
all  standards  which  are  to  be  established.  For  operational  systems, 
syste  o  descriptive  data  generally  exist;  the  major  task  is  that  of  extracting 
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information  from  reports  and  records  and  supplementing  that  with  ship¬ 
board  inspections  or  visits  to  training  centers.  Human  capability  data, 
necessary  for  integrative  use  of  the  method,  currently  are  not  widely 
available  in  data  store  form.  Until  they  are,  that  requirement  must  be 
filled  by  collecting  performance  data  empirically  on  subject  groups 
composing  a  specific  personnel  subsystem  design. 
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III.  METHOD  DEVELOPMENT 


The  purpose  of  the  study,  as  noted  previously,  was  to  develop  a 
method  that  would  fulfill  *rwo  objectives: 

.  Derivation  of  specific  personnel  performance  standards 
with  definable  relations  to  ultimate  system  effectiveness 
requirements,  and 

.  Determination  .f  the  effect  on  system  effectiveness  of 
performance  levels  that  deviate  from  established 
performance  standards. 

To  meet  those  objectives  it  was  required  to  develop  a  set  of  procedures 
and  analytic  or  probabilistic  tools  which  would  take  system  effectiveness 
requirements  and  system  design  data  as  inputs  and  provide  s*,  stem- 
related  personnel  performance  standards  as  outputs.  Since  effectiveness 
requirements  are  most  frequently  established  for  the  overall  operation 
of  the  system,  standards  derivation  --  the  first  methodological  objective  -- 
becomes  a  process  of  allocating  overall  system  requirements  among 
personnel/equipment  functional  units.  Essentially  the  same  method  must 
permit  determination  of  the  effect  of  a  given  personnel  subsystem  design 
on  system  effectiveness  to  achieve  the  second  methodological  objective. 

As  a  first  step  in  method  development,  the  conceptual  framework  was 
used  as  a  basis  for  establishing  major  procedural  steps  which  during  the 
course  of  the  study  were  ultimately  refined  to  the  following: 

1.  Definition  of  system  operational  requirements. 

2.  Definition  of  functional  unit  levels  ol  specificity. 

3.  Determination  of  relations  among  PEE  Units  through 
construction  of  graphic  and  mathematical  models. 

4.  Allocation  of  overall  requirements  to  subordinate 
elements  to  establish  standards  for  all  PEE  Units. 

5.  Utilization  of  the  mathematical  model  and  derived 
standards  to  determine  the  effect  of  a  given  per¬ 
sonnel  subsystem  design  on  ultimate  system 
effectiveness  requirements. 

Steps  1  through  4  treat  the  derivative  process  while  Step  5  is  related  to 
the  integrative  process. 
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Ill  the  sections  below,  the  approach  to  method  developme  nt  and  a 
description  of  the  major  steps  are  presented  with  a  description  of  the 
various  uses  of  kinds  of  input  data  listed  in  section  II-B. 

A.  Definition  of  System  Operational  Requirements 

System  operational  requirements  constitute  a  quantitative  description 
of  what  functions  the  system  must  perform,  and  how  well  it  must  perform 
them.  Since  they  represent  the  ultimate  scale  against  which  system  per¬ 
formance  is  measured,  they  are  the  logical  choice  as  the  basis  for 
establishing  objective  standards  for  personnel  performance. 

Specificity  in  definition  of  the  hierarchy  of  system  requirements  varies 
among  systems,  but  the  following  are  representative  of  those  required  to 
establish  job  standards: 

.  Overall  effectiveness  requir*  ments  (e.  g. ,  overall  probability 
of  accomplishing  the  mission  for  which  the  system  was 
designed) 

.  Performance  requirements  for  major  system  functions  (e.g., 
for  a  radar  system,  time  and  probability  requirements  for 
detection  and  tracking  would  be  included) 

.  Availability  requirements  (reliability  and  maintainability) 

A  s  MTBF/(MTBF+MTTR) 

where: 


MTBF  =  Mean  Time  Between  Failures 
MTTR  =  Mean  Time  to  Restore 

An  illustration  of  the  relation  among  the  various  requirements  for  a 
radar  system  follows: 

E  =  P(d)  •  P(a)  •  P(t)  •  A 


where: 

E  =  effectiveness 

P(d)  =  probability  of  detection  (e.g.,  probability  of 
detecting  a  Bm^  target  at  100  miles) 

P(a)  =  probability  of  altitude  determination  (e.  g. , 
probability  of  determining  altitude  within 
1,  000  feet) 
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P(t)  =  probability  of  tracking  (e.  g. ,  probability 
of  successfully  tracking  the  target) 

A  =  availability  (probability  that  the  syste^n 
will  be  operating  at  a  specific  point  in 
time). 

Additional  data  that  could  be  used  in  establishing  standards  would  be 
time  constraints: 


t 
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d 
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'd 

Td 


max 


t 

Ts 


max 


detection  time 

specified  maximum  detection  time 
altitude  determination  time 

track  determinat:  n  time 

specified  maximum  time  to  perform  all 
system,  functions. 


B.  Definition  of  Functional  Unit  Levels  of  Specificity 


Within  our  conceptual  framework  for  establishing  performance 
standards,  the  PEF  Unit  has  been  defined  as  a  man-machine  functional 
interaction  at  a  given  level  of  specificity.  At  the  most  gross  level  of 
specificity,  an  illustrative  PEF  Unit  might  be  "detection  of  a  target." 

At  a  more  elemental  level,  an  illustrative  PEF  Unit  might  be  "^election 
of  a  transmission  frequency  by  positioning  a  selector  switch."  A  PEF 
Unit  specificity  hierarchy  of  the  following  form  serves  as  a  useful 
framework: 

.  System  Function 
.  Personnel  Function 
.  Task 


-  15  - 


Subtask 


.  Element 

At  various  levels  of  analysis,  the  system  may  be  described  as  consisting 
of  X  system  functions,  or  y  tasks,  jor  z  elements.  While  the  PEF  Units 
at  the  extremes  of  the  hierarchy  have  been  reasonably  well  defined, 
definition  of  the  intermediate  levels  has  been  somewhat  hampered  by  the 
variability  of  PEF  Units  found  in  various  systems.  To  avoid  the 
problem  encountered  in  defining  intermediate  levels  and  testing  for  their 
existence  in  any  specific  system,  our  approach  has  been  to  identify  the 
system  function  PEF  Units  as  Level  I,  and  subordinate  PEF  Units  as 
Level  II,  Level  III,  etc.  At  Level  I,  system  functions  are  generally 
designated  by  a  brief  noun  title,  such  as  "detection,  "  "identification,  " 
or  "inspection."  Alternatively,  exclamatory  verbs  hold  equally  descriptive 
communicative  value,  "detect,"  "identify,"  or  "inspect."  Each  of  those 
general  categories  includes  PEF  Units  both  necessary  and  sufficient  to 
accomplish  the  system  mission.  In  addition: 

1.  The  system  function  is  always  immediately  related  to  the 
system  mission,  (i.e. ,  no  intermediary  level  of  system 
operation  exists);  and 

2.  The  system  function  is  developed  apart  from  and  without 
regard  to  the  specific  hardware  of  the  system  being 
described.  As  a  ^ale,  systems  with  similar  missions 
will  have  similar  functions. 


C.  Graphic  Description  of  Functional  Relations 


To  develop  an  objectr-e  method  of  deriving  system  performance 
standards,  a  general  approach  was  sought  by  which  a  mathematical 
model  of  any  system  could  be  constructed  to  describe  the  relations  be¬ 
tween  definable  PEF  Units.  It  was  recognized  that  before  an  accurate 
mathematical  model  could  be  produced,  system  information  would  have  to 
be  organized  to  show  meaningful  functional  relations  among  comparable 
PEF  Units.  As  a  result,  a  diagrammatic  modeling  technique  was 
developed.  Since  all  Personnel/Equipment  functions  effect  a  change  in 
the  state  or  status  of  the  system,  PEF  Units  are  graphically  defined 
according  to  the  states  of  the  system  just  prior  to  and  immediately 
following  their  occurrence.  The  technique  of  depicting  a  system  in  that 
manner  produces  a  diagram  which  we  t»^‘rm  a  Graphic  State  Sequence 
Model  ( GSSM) .  1 


The  GSSM  is  not  to  be  confused  with  an  operational  sequence  diagram 
(OSD)  which  stresses  human  decisions  and  actions  rather  than  personnel/ 
equipment  functions  which  are  defined  by  specific  configurations  of  input 
and  output  states.  _  If,  _ 


If  we  assume  that  the  criteria  for  acceptable  system  functioning  can 
be  fully  defined,  including  all  tolerances  and  relevant  constraints,  a 
sequence  of  states  can  be  generated  by  dividing  the  system  into  a  patterned 
series  of  spacio-temporal  units  of  performance  (or  PEF  Units)  so  that  one 
can  describe  completely  the  instantaneous,  unchanging  state  of  the  system 
between  each  unit.  That  is,  PEF  Units  are  delineated  by  the  static 
characteristics  of  the  system  at  specified  instants  in  time.  In  practice, 
however,  it  is  impractical  to  describe  all  features  of  the  system  between 
each  functional  unit,  particularly  when  there  are  many  such  units,  each 
producing  a  relatively  small  change  in  the  system.  Instead,  one  need 
describe  only  those  static  characteristics  relevant  to  each  single  PEF 
Unit.  As  indicated  in  Figure  1,  the  entire  system  can  be  viewed  as  a 
single  PEF  Unit;  the  input  state  defines  the  relevant  system  conditions 
immediately  prior  to  operation  initiation,  and  the  output  state  reflects  the 
mission  requirements  and  system  effectiveness  criteria. 


Figure  1.  The  system  as  a  single  PEF  Unit. 


Clearly,  the  very  simple  appearing  model  in  Figure  1  describes  the 
most  complex  personnel/equipment  function.  Conversely,  as  one  examines 
the  system  in  increasing  detail  and  descrioes  system  operations  in  terms 
of  more  specific  but  less  complex  PEF  Units,  the  GSSM  grows  increasingly 
complex.  For  example.  Figure  Z  is  an  illustration  of  a  Level  II  GSSM. 

In  that  figure  the  receive  and  record  activities  of  the  AN/SKQ-1  ^  are 
shown  to  comprise  three  functions,  I,  II  and  HI  (i.  e.  ,  three  Level  I  PEF 
Units),  each  of  which  has  been  elaborated  at  specificity  Level  II,  with 
PEF  Units  represented  by  the  letters  A  through  J.  Further  elaboration 
of  the  second  level  activities  would  result  in  a  GSSM  with  a  greater  number 
of  PEF  Units  and  more  complex  interrelations  between  them. 

However,  no  matter  how  derailed  the  GSSM,  one  must  begin  with  a 
complete,  objective  description  of  the  initial  input  state,  final  output  state 
and  constraints  as  depicted  in  Figure  i.  The  same  diagrammatic  approach 
is  applicable  in  treating  PEF  Units  at  any  level  in  the  specificity  hierarchy 


A  shipboard  equipment  for  receiving  and  recording  telemetered  signals 
frr  ri  Navy  surface-to-air  guided  missiles. 
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from  the  highest  (or  Level  1),  to  the  lowest  (or  element)  level  which 
involves  simple  personnel/ equipment  operations;  the  performance  standard 
method  will  generally  require  that  the  system  be  described  to  the  element 
level* 

Specific  procedures  for  constructing  a  GSSM  have  been  identified  and 
are  described  in  Appendix  B.  The  technique  utilizes  concepts  of  symbolic 
logic  commonly  applied  in  computer  design.  The  construction  of  a  GSSM 
for  the  AN/SPS-40  radar  presented  in  Volume  11  of  this  report  and  the 
model  segments  shown  in  the  following  sections  illustrate  the  general  use 
of  the  technique. 

'^u  •  struct  a  highly  descriptiv'e  GSSM  from  system  design  data 

uir  c.  t.-aining  and  special  caution.  Of  primary  importance  is  the 
fj  -cssst'  :o  approach  the  job  without  bias;  i.e. ,  the  GSSM  must  be 
dev^T.*  .•  such  that  it  is  minimally  affected  either  by  the  system  designer’s 
^caceut  uf  how  the  syst.  n  is  expected  to  operate  or  by  information  regard¬ 
ing  -he  past  performance  of  an  operational  system.  By  attendixig  primarily 
to  required  input  and  output  states  --  rather  than  to  the  intermediate 
processes  --  it  becomes  possible  to  consider  all  possible  ways  A  getting 
from  one  state  to  the  next.  The  technique  allows  a  detached  appraisal  of 
—tem  design  since:  (1)  unexpected  processes  may  emerge  as  real  and 
table  alternatives  to  the  one(s)  in  mded;  (2)  omissions  in  design  may 
ncovered  by  observing  the  inadequacy  of  a  state  for  initiating  a  PEF 
.*.-t;  and  (3)  design  weaknesses  stand  out  when  the  analyst  is  forced  to 
ask,  ’'V'hat  condition(s)  necessarily  give  rise  to  the  next  set  of  functions 
and  how  can  we  ascertain  their  occurrence?" 

A  significant  contribution  to  method  development  was  made  by  using 
the  graphic  modeling  technique  to  define  the  elemental  PEF  Umt.  By 
virtue  of  its  objectr*ity,  the  logic  diagram  eliminates  problems  usually 
associated  with  reliance  on  behavioral  data.  Thus,  it  is  possible  to 
specify  the  characteristics  of  the  elemental  PEF  Unit  according  to  non- 
behavioral,  relative  criteria.  The  complete  definition  is  presented  in 
Appendix  B. 

The  importance  of  delineating  meaningful  elements  of  system  per¬ 
formance  rec^fci  on  the  fact  that  human  activity  can  be  reduced  theoretically 
to  an  underlying  set  of  neuromuscular  and  mo’ecular  changes.  However, 
one  might  spend  much  time  trying  to  define  a  system  at  that  level  without 
ever  arriving  at  a  means  for  establishing  standards.  It  is  necessary, 
therefore,  to  state  system  behavior  only  to  that  desc^ibable  and  measurable 
level  necessary  and  amenable  to  an  effective  r.^ethodology.  In  our  approach, 
the  element  PEF  Unit  represents  that  minimum  level;  it  is  definable 
ac  "ding  to  non -behavioral,  relatively  ''bjective  criteria  relating  to  con¬ 
st'  :tion  by  logical  diagramming.  Any  further  attempt  at  moving  to  a 
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lower  level  would  have  no  effect  on  the  basic  structure  of  the  descriptive 
mathematical  model  we  wish  to  derive  from  the  GSSM  at  the  element 
level.  It  is  the  mathematical  model,  not  the  GSSM,  which  provides  the 
basis  for  determining  probabilities  (for  standards)  associated  with  system 
performance  requirements,  so  procedural  efficiency  is  unnecessarily 
abused  by  erecting  a  more  complex  GSSM  than  is  required  oy  the  mathe¬ 
matical  model  we  wish  to  derive.  However,  since  the  mathematical 
model  will  be  derived  directly  from  the  graphic  model,  the  latter  must 
be  complete  and  accurate  for  the  mathematical  statement  to  represent 
the  system  faithfully. 

D.  Mathematical  Statement  of  Fur.  .tional  Relations 


Once  a  system  design  is  defined,  a  Graphic  State  Sequence  Model  of 
it  can  be  constructed  to  a  desired  level  of  specificity.  If  we  assume  the 
system  functions  to  be  completely  described,  i.e.  ,  all  personnel  and 
equipment  operations  defined  for  every  meaningful  contingency,  we 
should  be  able  to  analyze  the  relations  between  the  probabilities  of  the 
output  states  of  all  the  PEF  Units  in  the  GSSM,  and  we  should  be  able  to 
derive  from  those  relations  a  single  system-descriptive  statement,  or 
Mathematical  State  Sequence  Model  (MSSM). 

For  an  understanding  of  the  state  sequence  analytical  procedure,  it  is 
necessary  that  the  system  be  observed  from  a  special,  impersonal  vantage 
point  from  which  the  defined  states  indeed  are  perceived  as  the  states  we 
talk  about,  and  not  any  others.  If  a  required  output  state  reads,  "the  three 
wing  nuts  in  positions  x,  y  and  z  are  tightened  so  that  at  least  25  inch-lbs 
torque  would  be  required  to  tighten  them  further,"  and  if  only  two  wing 
nuts  reach  that  criterion,  the  the  output  state  is  not  reached.  If  the 
operator  says  the  criterion  is  reached  but  it  isn’t,  his  saying  so  is  not  the 
system's  fault  or  concern;  the  system  has  failed.  Only,  for  the  latter 
condition,  the  failure  is  not  included  in  the  probability  statement  for  an 
output  state,  when  a  given  input  state  begins  the  PEF  Unit.  Assume  we 
know  that  20  percent  of  the  men  who  have  performed  this  unit  fail  to  screw 
the  wing  nuts  adequately,  but  they  act  as  if  the  criterion  is  met.  From  the 
system  performance  point  of  view,  the  output  state  exists  but  with  a 
probability  of  .  80  times  the  probability  of  the  output  state  due  to  other 
factors. 

Figure  3  shows  a  simple  PEF  Unit  with  a  binary  output  state,  either 
B  or  everything  else,  i.e.,  B  =  not  B.  System  performance  depends  upon 
the  occurrence  of  the  output  B.  In  the  example,  B  is  said  to  exist  if  the 
system  acts  as  if  the  three  wing  nuts  satisfied  the  tightening  conditions, 
and  from  our  hypothetical  observations,  the  system  correctly  interprets 
that  B  exists  80  percent  of  the  time.  If  the  system  erroneously  operates 


-  20  - 


A 


P(B)  +  P(:B)  =1.0 


Figvre  3.  The  simplest  GSSM  of  a  single  PEF  Unit. 


’inder  the  assumption  of  B  (that  is,  if  B  actually  exists,  but  it  is  not 
treated  as  such),  that  does  not  change  anything.  The  system  will  still 
not  function  properly  because  B  is  interpreted  as  having  occurred. 

In  the  development  of  the  mathematical  modeling  technique  it  was 
recognized  that  since  either  B  or  B  must  occur,  the  outpvat  states  m 
the  graphic  and  mathematical  models  could  omit  the  B  (and  all  analogous 
output  states)  without  loss  of  information.  Thus,  although  no  further 
reference  will  be  made  to  the  undesirable  output  states,  they  are  always 
understood  to  exist. 

We  may  now  consider  the  Mathematical  State  Sequence  Model  (MSSM) 
of  the  graphic  representation  of  a  single  PEF  Unit,  such  as  the  one 
illustrated  in  Figure  3.  Probability  theo»’y  tells  us  that 

P(B-A)  =  P(B/A)P(A)  =  P(A/B)P(B) 

where  P(B*A)  is  the  probability  of  the  joint  occurrence  of  B  and  A,  and 
P(B/A)  is  the  conditional  probability  that  B  occurs  given  that  A  exists. 
Similarly  P(A/B)  describes  the  degree  to  which  A  occurs  given  that  B 
exists.  But  if  B  exists,  A  must  have  occurred  --  from  the  system's 
point  of  view  --  because  B  results  from  A.  We  must  assume,  initially 
at  least,  that  the  system  has  been  des  gned  to  arrive  at  its  specified 
output  state  via  particular  definable  procedures,  so  if  state  B  exists, 
state  A  must  have  existed  at  the  beginning  of  the  PEF  Unit,  else  the 
PEF  Unit  would  not  have  occurred. 

In  reality,  it  is  conceivable  that  within  the  generalized  system  context, 
state  B  rr.ight  seem  to  occur  without  prior  A.  But,  the  system  would  then 
perceive:  (1)  that  its  design  failed  to  consider  all  possible  alternatives  of 
A,  (2)  that  B=A  or  some  prior  failure  condition,  or  (3)  that  occurrence 
of  B  is  a  fortuitous,  transient  condition  which  is  part  of,  but  not  relevant 
to,  the  ongoing  operations. 


GSSM  symbols  are  defined  in  Figure  2, 
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Thus,  from  the  system  point  of  view,  P{A/B)=  1.0,  so  that  P(B)  = 
P(B/i^P(A),  and  since  A  represents  the  output  state  (or  a  portion)  of  a 
prior  PEF  Unit,  P(A)  may  be  derived  in  a  similar  manner.  The 
procedure  can  be  repeated  back  to  the  initial  input  state  of  the  system. 

In  man-machine  systems,  certain  forms  of  P(B/A)  are  often  observed 
to  be  partitioned.  For  example,  (still  referring  to  Figure  3)  assume 
that  A  =  "power  is  supplied  to  VTVM  and  ON /OFF  switch  has  been  ON 
position  for  five  minutes  or  more",  ^  and  B  =  "VTVM  indicator  needle 
reads  zero  volts."  The  PEF  Unit  is,  therefore,  a  simple  calibration 
procedure. 

The  system  requirement  states  that  the  needle  reads  zero  volts  when 
no  voltage  measurement  is  taken;  how  one  gets  there  is  of  no  concern  to 
the  system.  To  illustrate  a  possible  complication  implied  by  some  PEF 
Units,  assume  that  we  observe  many  operators  using  the  VTVM  under  the 
conditions  of  this  system  (or  a  similar  one).  We  might  find,  for  example 
that: 

1.  Given  the  input  state.  A,  0.60  of  the  men  actually  do  calibrate 
the  VTVM,  while  0.40  do  not.  [KisO.bO;  K2  =  0.40] 

2.  0.50  of  the  time,  the  VTVM  is  accurately  calibrated,  i.e., 

B  already  exists  without  any  adjustment  required. 

[P(B/A2)  =  0.50] 

3.  0.  95  of  the  time  when  the  adjustment  is  made,  the  oporator 
achieves  the  output  criterion.  [P(B/A2)  =  0.95] 

The  hypothetical  observations  imply  that  the  system  required  output 
state  may  exist  (50  percent  of  the  time)  without  an  operator’s  doing  any¬ 
thing.  Since  Figure  3  does  not  allow  for  that  possibility,  it  is  incomplete; 
instead,  the  correct  GSSM  would  appear  as  showii  in  Figure  4. 

The  symbols  (Aj)  and  (A^)  designate  the  alternative  means  by  which 
state  B  may  be  reached.  P(B/A2)  is  interpreted  as  follows;  given  that 
power  is  supplied  to  the  VTVM  and  the  ON/OFF  switch  has  been  in  the 
ON  position  for  five  minutes  or  more,  P(B/A2)  is  the  probability  that 
the  VTVM  is  accurately  calibrated.  Thus,  state  A^  is  state  A.  The 
same  is  true  for  state  Ai. 


Omitted,  for  the  sake  of  brevity,  are  such  requirements  as  "electrodes 
are  in  contact  only  with  free  air  and  selectfjr  switch  is  in  "  (a 
particular  voltage  position). 


22  - 


K 


(A2) 


(Ai) 


CALIBRATE 

VTVM 


Kz 


B 


Ki  +  K2  =  1.0 


P(B)=KiP(B/Ai)P(Ai)  +  K2P(B/A2)P(A2) 


(These  are  the  proportions 
respectively  established  for 
alternative  procedures  to 
arrive  at  state  B.) 

(The  different  conditional 
probabilities  reflect  the 
effects  of  the  different 
procedures. ) 


Note;  The  subscripts  differentiate  between  the  mutually 
exclusive  means  of  getting  from  state  A  to  state  B; 
so  that,  state  A  =  state  Aj  =  state  Ai  or  alternatively, 
P(A)  S  P(Ai)  5  P(A2). 

Therefore: 

P(B)  =  [K jP(B/Ai)  +  K2P{B/A2)]  P(A) 


Figure  4.  GSSM  and  MSSM  of  a  PEF  Unit  with  the  alternative 
condition  that  it  is  not  executed.  1 


The  MSSM  of  Figure  4  can  be  solved  using  the  observed  data  we 
hypothesized  in  the  example  above; 

P(B)  =  [KiP(B/Ai)  +  K2P(B/A2)]  P(A) 

P(B)  =  [.  60  (.  96)  +  .  40  (.  50)]  P(A)  =  [.  54  +  .  20]  P(A) 

P(B)  =  0.  74  P(A) 

If,  now,  the  system  effectiveness  requirement  tolerances  lead  to  the 
conclusion  that  the  VTVM  must  be  calibrated  correctly  at  least  0.  80  P(A) 
of  the  time,  something  must  be  done  to  modify  the  system  so  as  to  ensure 
that  it  meets  that  criterion;  e.g. ,  we  might  specify  that  Ki  be  increased 
through  improved  operator  training. 


GSSM  symbols  are  defined  in  Figure  2. 
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When  several  mutually  exclusive  choices  exist,  the  resulting  MSSM 
is  an  extension  of  the  one  shown  in  Figure  4.  The  general  model 
expressing  n  such  choices  of  means  of  changing  from  state  A  to  state  B  is 

n 

P(B)  =  P(A)  2  Kx  P{B/Ax) 
x=  1 


where  Kx  is  the  proportion  of  the  time  (or  of  the  total  number  of  operations) 
that  B  is  reached  via  the  alternative  symbolized  by  A^;  if  (B/A)  occurs  at 
all,  then  K  \  +  . . .  +  Kjj  =1.0. 

Consider  the  example  of  a  man  determining  the  probability  that  he 
will  drive  home  from  work  within  30  minutes.  If  he  can  (and  sometimes 
does)  choose  among  several  routes,  he  could  calculate  the  relative 
frequency  with  which  he  takes  each  of  them.  And  he  could  --  with  a  large 
enough  sampling  --  determine  for  each  route,  the  probability  that  he  will 
arrive  at  home  within  the  time  limit.  Quite  possibly  there  will  be  one 
route  which  gets  him  home  by  30  minutes  more  often  than  any  of  the  others. 
Clearly,  if  he  does  not  always  take  that  best  route,  the  probability  that  he 
meets  his  criterion  will  be  less  than  if  he  did. 

The  example  illustrates  the  principle  that  the  probability  of  an  output 
state  decreasves  with  an  increase  in  the  number  of  choices:  (1)  if  the 
probabilities  of  the  choices  differ,  and  (2)  if  the  relative  frequency 
associated  with  the  selection  of  a  new  alternative  is  split  from  one  with 
higher  relative  frequency,  and  nothing  is  done  to  increase  the  likelihood 
of  selecting  a  better  alternative. 

The  implications  here  for  system  design  are  apparent:  when  a  high 
probability  of  a  particular  output  state  is  required  and  an  effective  and 
efficient  procedure  is  known  to  exist,  the  design  must  preclude  departures 
from  that  procedure.  The  implications  for  model  constructi^'n  are  almost 
as  obvious:  the  GSSM  must  include  all  meaningful  possibilities  --  even  if 
not  explicitly  stated  in  operator  instructions  --  else  the  MSSM  will  yield 
erroneous  results  and  conclusions. 

Inspection  of  several  system  designs  in  GSSM  form  has  led  to  the  find¬ 
ing  that  the  alternative  arrangement  of  PEF  Units,  described  above,  is 
one  of  two  common  configurations.  The  second  type  may  be  called 
"summary,  "  which  refers  to  arrangements  of  required  PEF  Units  for  which 
no  alternative  exists.  Figure  5  illustrates  two  ways  in  which  the  summary 
configuration  may  appear:  series  (A  to  a  to  b)  and  parallel  (b  to  c  and 
b  to  d). 
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A 


B  =  c  plus  d 
(or  c,  d) 


P(B)  =  P(c.d/b)  P(b) 

P(B)  -  P(c/b)  P(d/b)  P(b) 
P(b)  =  P(b/a)P(a) 

P{a)  =  P(a/A)  P{A) 


Therefore, 

P(B)  =  P(A)  P(a/A)  P(b/a)  P(c/b)  P(d/b) 


Figure  5.  Summary  configurations  --  required  series  and 

parallel  PEF  Units. 


The  resulting  MSSM  for  the  summary  configuration  is  a  product  of  all 
conditional  probabilities.  Stated  in  general  form  for  n  PEF  Units 

n 

P(B)  =  P(A)  n  P(o^/i^) 

a  =  1 

where  Oq  is  the  output  state  of  the  a  th  PEF  Unit  and  1^  is  its  input  state. 

The  MSSM  for  the  AN/SKQ-l  receive  and  record  activities,  shown  in 
Figure  2,  combines  the  alternative  and  summary  configurations.  In 
that  figure,  (1)  if  j  and  Kg^  were  omitted,  (2)  if  we  let  x  represent  the 
state,  At  <20  sec.  and  y  represent  "command  to  start  recording,  "  and 
(3)  if  P(A)  is  used  to  represent  the  conditional  probability  of  the  output 
state  of  A  given  its  input  state,  then  without  alternative  condition.^  the 
GSSM  would  be  purely  summary  in  form,  ani  the  MSSM  would  be 

J 

P(output  state)  =  P(x)  P(y)  n  (a  ) 

a  =  A 
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When,  however,  the  two  alternative  situations  are  included,  the  MSSM  is 

P(output  state)  =  P(x)P(y)P(A)P(B)P(E)P(F)P(H)P(I)P(J)[KclP(C)  + 

Kc2P(C;  )[KgiP(G)+Kg2P(G)  ] 

where  P(C)  and  P(G)  represent  the  probability  of  the  existence  of  the  out¬ 
put  state  without  the  occurrence  cf  C  and  G,  respectively.  While  most 
mathematical  models  required  for  the  MSSM  are  summary  and  alternative 
types,  one  additional,  rather  infrequently  used  type  has  been  found  for 
repetitive  functions  designed  to  increase  the  accuracy  of  an  output  state. 
An  example  of  this  condition  is  shown  in  Figure  6. 

Consider,  for  example,  the  requirement  that  an  estimate  of  the  size 
of  a  distant,  moving  object  be  made  within  a  specified  tolerance  with  a 
given  probability  of  accomplishment.  If  it  can  be  assumed  that  with  each 
repetition  --  performed  so  as  to  improve  over  each  preceding  estimate  -- 
the  probability  of  estimating  within  the  tolerance  level  increases  by  a 
constant  proportion,  k,  then  the  mode  could  be  expressed  as  follows: 

Assuming  an  exponentially  decreasing  effect  of  repetition 
(because  the  probability  of  success  cannot  be  greater  than 
unity),  then 


P(B}jn=  1 -Ce  1 -[1 -P(B)]e  m=l,2,3...  (1) 

where  P(B)jyx  is  the  output  probability  after  m  repetitions  and 
P(B)  is  the  output  probability  with  no  repetitions,  i.e.,  m  =  0. 

(See  Figure  6. ) 

Where  m=l,  P(B)jn  =  P(B)  {I  +  k} 

Therefore,  after  solving  equation  (1)  for  c, 

P(B)^=  1  -  [1  -  P(B)]exp.  [m  In 

1  -  P(B) 


which  can  be  simplified  to 

P(B)jn  =  1  -  [1  -  P(B)]1  -  (1  -  P(B)  { 1+  k  (2) 

If  k  is  not  constant,  but  rather  varies  as  a  function  of  m,  the  number 
of  repetitions,  then  that  function  would  be  substituted  for  k. 


-  26  - 


A 
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Figure  6.  Example  of  a  GSSM  for  improving  an  estimate 

by  repetition. 


It  should  be  noted  that  the  validity  of  equation  (1)  remains  to  be  tested. 
That  can  be  done  by  observing  how  the  probability  of  accomplishing  each 
of  a  number  of  relevant  tasks  changes  as  a  function  of  the  number  of 
repetitions  allowed  for  improving  performance  of  reaching  a  specified 
level  of  excellence. 

The  three  basic  mathematical  models  described  above  cover  all 
system  configurations  that  have  been  examined  to  date.  It  appears  very 
likely  that  most,  if  not  all,  operations  can  be  described  by  one  or  a 
combination  of  those  three.  Once  the  MSSM  for  the  system  has  been 
obtained,  it  then  serves  as  the  foundation  upon  which  allocation  is 
performed. 

E.  Allocation  of  Requirements 


Allocation  is  the  process  of  distributing  system  operational  require¬ 
ments  among  the  system  PEF  Units.  The  allocated  requirements  which 
appear  as  components  of  the  output  states  thus  become  the  standards 
which  must  be  met  in  order  to  achieve  the  overall  system  criteria  for 
a  successful  mission. 
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1.  Problem  Definition 


Ideally,  the  data  used  in  the  allocation  process  should  be  solely 
restricted  to  system  effectiveness  requirements,  supplemented  when 
appropriate  with  hardware  design  data.  However,  system  requirements 
may  not  provide  directly  for  establishment  of  standards  at  every  level 
of  specificity,  particularly  at  the  elemental  level.  Therefore,  to  arrive 
at  those  standards  from  system  effectiveness  requirements,  it  is 
necessary  (1)  to  establish  a  valid  distribution  rule  and  (Z)  to  determine 
the  nature  of  the  data  to  which  the  rule  must  apply.  The  best  results 
would  be  obtained  if  those  data  were  reliably  measurable  and  not  dependent 
upon  any  specific  system  configuration.  However,  no  objective,  independent 
basis  has  yet  been  developed  for  determining  generalizable ,  consistent 
human  performance  measures  which  are  not  system-specific.  And  -- 
as  will  be  discussed  in  a  later  section  --  what  is  available  in  the  form  of 
capability  data  leaves  much  to  be  desired. 

Before  considering  the  procedure  developed  to  apply  those  data 
in  a  distribution  rule,  some  initial  comments  will  be  made  regarding 
several  important  considerations  in  deriving  that  rule. 

Use  of  mission  requirements  and  system  data  will  most  frequently 
lead  directly  to  establishment  of  standards  for  Level  I  and  perhaps  Level  II 
PEF  Units.  Application  of  the  job  standards  methodology  developed  to 
date  would  p- ovide  system  related  standards  for  those  levels.  A  require¬ 
ment  remains,  however,  to  develop  more  detailed  standards  --  specifically 
to  the  element  level  --  for  reasons  including  the  following: 

.  The  absence  of  standards  at  lower  levels  does  not  permit 
a  test  of  the  "reasonableness”  of  individual  assignments 
which  might  suggest  reallocation  of  requirements  (when 
this  is  possible  within  the  mission  context).  A  test  of 
"reasonableness"  here  would  consist  of  a  gross  com¬ 
parison  of  assigned  requirements  and  feasible  human 
performance. 

.  The  majority  of  our  tools  for  measuring  or  evaluating 
personnel  performance  are  either  insensitive  or  in¬ 
efficient  at  levels  much  higher  than  the  element  level, 
and  job  standards  at  gross  levels  are  difficult  to  treat. 

As  an  example,  determining  the  ability  of  a  person  to 
perform  the  detection  function  might  entail  a  complex 
simulation  arrangement,  and  if  substandard  per¬ 
formance  were  noted,  a  detailed  analysis  of  elemental 
PEF  Units  would  most  likely  be  required  to  determine 
where  the  deficiencies  lie. 
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.  Because  personnel  functions  and  tasks  vary  significantly 
from  system  to  system,  methodological  requirements 
dictate  that  the  "basic  building  block"  of  performance  be 
small  enough  to  permit  universal  identification  across 
systems. 

.  Transitive  use  of  the  method  to  determine  the  effect  of 
varying  levels  of  personnel  performance  as  well  as  use 
of  the  method  for  evaluating  occupational  structure 
appears  to  require  that  standards  be  defined  at  the 
element  level. 

Thus,  standards  associated  with  all  PEF  Units  will  be  constrained, 
if  not  directly  established  by  system  effectiveness  requirements.  However, 
the  reed  to  allocate  to  the  element  level  will  require  the  use  of  data  external 
to  the  system,  and  the  most  obvious  alternative  is  employment  of  behavioral 
data  of  some  type.  There  are  several  ways  in  which  the  allocation  may  be 
accomplished,  among  them  the  following; 

1)  Use  of  an  approach  which  establishes  a  proportional 
relation  among  individual  PEF  Units  using  behavioral/ 
systems  experts  (human  factors  analysts)  or  specific 
systeiiis  experts  (design  engineers  or  technicians) 
as  judges. 

a)  Ranking 

(1)  According  to  specific  dimensions  (e.  g. , 
according  to  expected  errors  in  performance), 
or 

(2)  According  to  a  general  dimension  (e.  g,  ,  an 
assessment  of  overall  "difficulty"  in  carrying 
out  the  task). 

b)  Rating 

(1)  Rate  individual  tasks,  elements,  etc...  accord¬ 
ing  to  specific  dimensions  .  .  . 

(a)  Single  dimensions 

(b)  Multiple  dimensions  which  would  require 
the  development  of  weighting  scores. 

(2)  Rate  individual  tasks,  elements,  etc.  according 
to  a  general  dimension. 
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2)  Use  of  experimental  data  from  similar  or  related  systems 
to  determine  element  PEf  Unit  standard  allocations  v^hicb 
will  satisfy  overall  effectiveness  requirements. 

a)  Use  directly 

b)  Use  to  establish  proportional  relations 

3)  Use  of  normaliiied  or  pooled  experimental  data  found  in 
recently  developeO  "data  stores"  to  determine  the  proper 
combination  of  elemental  PEF  Unit  standards  which  will 
satisfy  overall  effectiveness  requirements. 

a)  Use  directly 

b)  Use  to  establish  weighted  indices 

4)  Use  of  simulation  study  data  obtained  through  system  specific 
sim«’lat:.on  to  determine  the  combination  of  element  PEF 
Unit  standard.*?  which  will  satisfy  overall  effectiveness 
requirements . 

Each  of  the  approaches  above  has  seme  merit  when  ev.aluated 
against  criteria  including: 

.  Reliability 

.  Validity 

.  Accuracy 

.  Sensitivity 

.  Objectivity 

.  Inc lusiv^e ness  (comprehensivene.'.s) 

.  Internal  Consistency 

.  Ease  of  Development 

.  Ease  of  Use 

The  sets  of  approaches  including  ranking  and  those  involving  scoring  by 
specific  syste-T".  experts,  however,  have  been  eliminated  due  to  limitations 
in  reliability  and  objectivity,  respectively.  Use  of  experimental  or 
simulation  study  data  appears  to  represent  a  ureful  approach,  but  the 
quality  cf  avaUable  data  will  vary  according  to  the  type  of  system  being 
analyzed;  problems  of  internal  consistency  may  aj.so  ex*ct.  Use  of 
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normalized  experimental  data  and  multi -dimensional  scoring  by  behavioral/ 
systems  experts  has  received  preatest  attention.  That  approach  is  described 
in  some  detail  in  the  following  section. 

2.  An  Index  Development  Approach 

A  ocoring  method  which  appears  quite  promising  for  use  in 
allocation  consists  of  developing  an  index  for  the  probability  of  accomplish¬ 
ing  a  P£F  Unit.  It  is  the  goal  of  this  approach  to  be  able  to  derive  a 
numerical  Index  of  Task  Accomplishment  (IOTA,  or  simply  Ig.)  for  all  man- 
machine  operations  based  on  a  function  of  the  variables  relevant  to 
effective  performance.  At  the  same  time,  the  means  by  which  the  IOTA  is 
derived  should  be  as  straightforward  as  possible  so  as  to  be  obtained 
reliably  by  any  systems  analyst. 

The  initial  task  is  to  define  the  relevant  variables;  the  following 
have  been  identified  as  at  least  some  of  those  which  may  contribute  to 
the  Index  (with  parenthetically  indicated,  possible  scoring  categories): 

A  =  the  number  of  human  acts^  in  a  PEF  Unit 
and  for  each  act  ( 1 ,  2,  .  .  . ,  m) 

C  =  continuity  (i.e.,  discrete  or  continuous) 

H  =  human  factors,  man-machine  interface 
design  (e.g.,  best  possible,  acceptable, 
or  poor  design) 

T  =  training  required  beyond  high  school  gre  luation 
(e.  g.  ,  detailed  instruction  and  supervi  ed 
practice,  simple  display  or  verbal  instruction, 
no  training) 

P  =  practice  or  current  skill  (e.g.,  recent  practice 
n  cessary,  some  practice  at  any  time,  no 
practice  necessary) 

D  =  level  of  discrimination  (e.  g.  ,  personnel  activity 
based  on  more  than  one  system  feature  change  or 
activity  based  on  only  one  system  feature  change) 

R  =  repetitiveness  (e.  g.  ,  much  repetition,  some 
repetition,  no  repetition) 

An  "act"  will  be  defined  more  fully  a"  the  method  is  developed.  For 
the  present,  consider  the  examples  "observe  ...  ",  "adjust,  .  .  "  and 
"swTtch  to  ..." 
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While  the  list  may  not  be  complete,  it  indicates  the  kinds  of 
general  variables  which  may  enter  into  index  determination. 

It  will  be  noted  that  most  of  the  factors  can  vary  on  a  continuous 
scale;  however,  in  order  to  maximize  reliability  among  indices  obtained 
by  different  analysts,  each  continuum  is  divided  into  two  or  three  large 
segments  (dichotomized  or  trichotomized).  By  accompanying  each 
segment  with  several  examples,  a  high  level  of  agreement  may  be 
expected  between  scores  on  the  resulting,  overall  IOTA.  That  is,  since 
the  measures  are  gross,  isolated  differences  between  scores  on  a 
particular  variable  will  have  a  relatively  small  effect  on  the  overall 
IOTA  because  many  such  gross  values  are  included  m  index  determination. 

Once  the  analyst  (or  scorer)  has  assigned  a  value  to  every  PEF 
Unit  on  each  variable,  the  overall  I^  must  be  calculated  by  inserting  the 
obtained  values  in  an  equatirin  describing  the  relation  among  the 
variables.  If,  for  example,  the  seven  listed  above  are  the  primary 
relevant  variables, 

I^  =  f(A.  C.  H,  T,  P,  D.  R) 

The  function,  itself,  must  be  determined.  To  find  the  nature  of  the  function 
and  the  constants  within  the  function  requires  that  applicable,  experimentally 
derived  objective  data  be  available. 

If  the  assumption  holds  that  the  IOTA  can  be  applied  to  all 
sonnel/ equipment  funct’onal  interactions,  then  behavioral  data  stores  of 
the  type  developed  by  Payne  and  Altman  offer  the  required  da‘-a.  ^  While 
there  are  certain  limitations  to  the  direct  use  of  data  for  allocating 
requirements  for  a  specific  systerr  ,  as  indicated  in  the  following  section, 
the  pooled  data  characteristic  can  be  used  advantageously  to  determine 
the  function  defining  the  IOTA.  To  illustrate  the  approach,  assume  that 
a  linear  relation  exists  between  the  IOTA  and  the  variables  relevant  to 
effective  performance: 


la  =  Ki  +  K2A  +  K3C+  K4H  ^  K5T  -  K^P  +  KyD  T  KgR 

A  large  sample  of  representative  PEF  Units  would  be  defined  and 
scored,  and  a  corresponding  estimate  of  1^  (i.e.  ,  error  probability)  for 
each  PEF  Unit  would  be  obtained  from  the  data  store.  Multiple  linear 


Payne  D.  ,  and  Altman,  J.  W.  An  Index  of  Electronic  Equipment 
Operability:  Data  Store.  Pittsburgh,  Pennsylvania:  American 
Institute  for  Research  Report  C-43-  1/62-FR,  January  1962. 
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regression  analysis  could  then  be  used  to  determine  values  for  the  various 
K' s.  Forms  of  the  IOTA,  other  than  linear  could  also  be  tested  to  deter¬ 
mine  the  best  form  of  the  expression.  Utilizing  existing  computer 
programs,  a  large  number  of  forms  could  be  tested  quickly  and  economically. 

Given  the  functional  relation,  an  IOTA  for  each  PEF  Unit  of  a 
particular  system  can  be  calculated.  The  next  problem  is  to  convert  1^ 
values  into  system- specific  probability  of  accomplishment  values  such 
that  the  probabilities  relate  to  system  effectiveness  criteria  in  accordance 
with  their  individual  contributions  to  the  system.  Since  the  MSSM  defines 
that  relation,  the  model  must  be  satisfied  by  some  probability  function 
of  the  I^  values.  To  accomplish  that,  the  following  tentative  assumptions 
are  made  for  exploratory  purposes: 

1)  The  la  values  lie  on  an  interval  scale  of  measurement. 

2)  The  la  values  can  be  constrained  to  he  between  zero 
and  1.0. 

3)  The  transformation  frum  the  IOTA  to  a  probability  of 
accomplishment  must  reflect  the  relation  that  the 
magnitude  of  the  increment  in  human  capability 
necessary  to  change  the  probability  of  accomplishment 
increases  as  the  nrcbability  of  accomplishment 
approaches  1 . 0. 

The  second  assumption  makes  it  possible  to  allocate  probabilities 
of  accomplishment  by  treating  the  dilferences  between  those  probabilities 
and  their  associated  indices  according  to  an  established  principle  for 
evaluating  increments  in  human  performance.  Since  the  IOTA  still  needs 
to  be  developed,  it  might  reasonably  be  expected  to  satisfy  the  second 
assumption  directly  by  using  it  as  one  of  the  criteria  in  tne  developmental 
process. 


The  third  assumption  suggests  the  hypothetical  transformation 
relation  shown  in  eq^^rition  (3). 


(l.O-Ia)-(l.Q-P)  P-Ia  _  . 

i.O-Ia  "  1.0 -la 

where: 

P  =  the  probability  component  of  a  performance, 
standard  (probability  of  accomplishment);  the 
P  for  eacl'  PEF  Unit  is  derived  from  its 
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associated  value.  Note  that  P  is  a  short¬ 
hand  notation  for  the  conditional  probability 
previously  referred  to  as  P{Oa/Ia)- 

k  =  a  constant 


Solving  equation  (3)  for  P, 

P  =  k{l-Ia)  +  Ia  (4) 

To  satisfy  any  system-required  probability,  P^,  involving  n 
PEF  Units, 


Po  =  f(Pl.  P^. 


Pn^  (k,  la]  *  ^a2' 


‘a„) 


(5) 


where  the  function,  f,-  determined  by  the  related  MSSM.  For  example, 
if  the  MSSM  is 


P„  = 


3 

n 

a  =  1 


then. 


Po=(l'(l  -lal)  *Ia,)(Ml  +  -Ia3)+la3] 

After  solving  equation  (6)  for  k,  each  P^  can  be  found  by  equation  (4). 
Those  Pq  values,  then,  are  the  required  probabilities  of  success  for 
each  PEF  Unit  in  order  for  the  system  to  meet  its  effectiveness  criterion. 

j.  Interi.71  Allocation  Technique 


In  order  to  provide  an  interim  allocation  technique  that  would  ful¬ 
fill  the  needs  cf  a  basic  capability  for  standards  derivation,  the  use  of 
normalized  or  pooled  experimental  data  to  establish  relative  weightings 
among  elemental  PEF  Units  was  selected  as  a  means  for  defining  lOIA 
(see  approach  3b  noted  previously). 

In  our  review  of  related  research,  the  most  comprehensive  set 
of  behavioral  data  found  was  the  "data  store"  developed  by  Payne  and 
Altman.  ^  The  purpose  of  their  study  was  to  provide  a  procedure  for  the 
quantitative  evaluation  of  electronic  equipment  operability.  Experimental 


Payne  and  Altman,  op.  cit.  1962. 


-  34  - 


information  pertaining  to  measurable  performance  in  the  operation  of 
electronic  equipment  was  abstracted  from  the  literature  and  organized 
into  a  data  store  containing  time  and  reliability  (i.e.  ,  probab.lity  of 
successful  accomplishment)  data.  It  is  intended  that  the  data  to  be  used 
to  score  electronic  equipment  along  time  and  reliability  dimensions  -o 
determine  its  acceptability  or  to  determine  redesign  alternatives  which 
would  increase  its  acceptability. 

The  major  limitations  in  using  the  "data  store"  pertain  to  task 
taxonomic  structure,  reliability,  consistency  and  comprehensiveness 
Concerning  task  taxonomic  structure,  a  significant  difference  was  noted 
between  the  one  found  most  appropriate  in  job  standards  derivation  methods 
and  the  one  employed  in  the  "data  store."  Specifically,  the  elemental 
PEF  Unit  has  been  rigorously  defined  through  the  use  oi  the  Graphic  State 
Sequence  Model,  whereas  the  entries  in  the  "data  store"  include 
fractionation  to  "partial  elements"  which  necessitated  a  translation  of 
"data  store"  terminology  to  conform  with  that  of  the  standards  derivation 
method,  and  vice-versa.  While  the  translation  was  made,  a  certain 
amount  of  descriptive  precision  was  necessarily  lost.  The  necessity  for 
translation,  of  course,  introduces  a  possible  source  of  user  error.  No 
statistical  reliability  index  was  computed  in  the  present  study;  however, 
in  a  trial  application  of  the  data  store  carried  out  by  Payne  and  A.ltman, 
a  range  of  agreement  among  raters  from  64  to  89  percent  was  observed. 

Problems  of  consistency  and  comprehensiveness  are  attributable 
to  the  shortage  of  well-documented  behav  loral  data  in  the  research 
literature.  Consistency  is  absent  in  the  conditions  under  v.  ..ic  h  the 
behavioral  data  were  collected,  size  of  the  Sdunple  of  observations  available 
to  corr.pute  time  and  reliability  statistics,  constraints  plaf-^d  on  performance, 
and  the  criteria  employed. 

Lack  of  compreh.ensiveness  refers  to  the  extent  to  wt'ich  *he  "data 
store"  omits  important  elements  cf  pe r sonnel/equipnient  oehavior.  To 
improve  comprehensiveness  is  a  two-fold  task:  first,  a  more  careful 
review  of  the  existing  research  literature  is  needed  to  meet  imn'.eO.ute 
requirements,  and  second,  increased  emph<:-is  must  be  placed  on  focusing 
upon  a  basic  framework  (e.  g.  ,  PEF  Un’t  description  principles)  in 
collection  of  behavioral  data  in  the  future.  The  latter  task  is  an  extensive 
but  important  one,  if  a  meaningful  and  complete  set  of  data  is  to  be 
established. 

.A.lthough  the  "data  store"  has  certain  significant  limitat'ons  with 
respect  to  its  use  in  allocation,  reliability  data  from  the  store  were  used 
as  indices  of  task  accomplishment  in  trial  application  of  the  method  to 
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the  AN/SPS-40.  Its  use  was  predicated  for  the  most  part  on  the  fact  that 
it  constitutes  the  most  complete  set  of  behavioral  data  classified  according 
to  tasks  normally  encountered  in  weapon  system  operation  and  maintenance. 
However,  due  to  some  of  the  limitations  noted  caution  must  be  employed 
in  use  of  "data  store"  information.  In  the  AN/SPS-40  application,  the 
reliabilities  were  not  taken  at  their  absolute  values,  but  rather  were  used 
to  scale  the  various  PEF  Units.  In  other  words,  the  "data  store" 
reliability  figures  were  treated  as  though  they  were  a  fully  developed  set 
oi'  Indices  of  Task  Accomplishment.  Therefore,  since  the  values  are  used 
principally  to  determine  PEF  Unit  combinational  weights,  it  is  not 
necessary  to  make  any  assumptions  regarding  the  accuracy  of  the  reliability 
valu'  S  themselves. 

In  summary,  it  was  found  that  at  least  one  set  of  data  could  be  used 
successfully  to  establish  Indices  of  Task  Accomplishment  and  allocate 
st<mdards:  however,  due  to  limitations  noted,  future  applications  of  "data 
store"  indices  must  be  made  carefully  and  tentatively.  The  approach  for 
arriving  at  a  more  reliable  and  comprehensive  basis  for  allocation  has  been 
described  above.  To  provide  an  objective  means  of  allocation  applicable 
to  a  wide  spectrum  of  man-machine  systems,  effort  should  be  directed 
toward  developing  the  approach  as  well  as  toward  close  examination  of  the 
limitations  of  existing  data  stores. 

F.  Integrative  Use  of  the  Method 


In  Section  I-A,  it  was  noted  tliat  the  method  must  provide  a  means  for 
estimating  the  way  system  operation  would  be  affected  by  deviations  from 
established  personnel  performance  standard-v.  This  section  explores  the 
integrative  use  of  the  method  for  that  purpose. 

Once  standards  have  been  allocated  to  system  PEF  Units,  the  question 
arises  as  to  how  well  those  standards  can  be  met.  Two  primary  concerns 
are  (a)  interpreting  results,  and  (b)  obtaining  indications  of  what  to  do 
about  discrepancies.  Since  the  fundamental  goal  of  the  technique  is  to  aid 
in  the  development  of  the  most  effective,  feasible  system  to  accomplish 
the  requirements  of  the  mission  assigned,  it  is  necessary  to  find  the 
optimal  match  between  personnel  performance  levels  required  by  a 
particular  system  design  (standards)  and  the  performance  levels  that  can 
be  expected  from  personnel  being  considered  for  assignment  to  that  system. 

For  the  method  to  be  employed  for  that  purpose,  some  form  of  per- 
sonci^l  capability  or  expectancy  data  are  required,  i.e.  ,  Sq  input  data 
for  tiie  integrative  model  included  in  the  conceptual  framework  (see 
Section  II-B).  At  present,  unfortunately,  no  comprehensive  and  fully 
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reliable  store  of  human  performance  capability  data  is  available.  In  the 
absence  of  behavioral  data  store  expectancies,  however,  capability  data 
can  be  obtained  on  the  specific  personnel  group  being  considered  for 
assignment  to  the  system  by  measuring  the  performance  of  that  group  on 
the  PEF  Units  resulting  from  the  standards  derivation  process.  Another 
source  would  be  proximate  data  obtained  from  previous  studies  of  per¬ 
sonnel  operations  that  are  comparable  to  those  required  by  the  system 
under  analysis.  A  less  adequate  alternative  is  to  use  the  ''data  store" 
developed  by  Payne  and  Altman,  i  That  alternative  was  selected  in  the 
present  study  since  the  "data  store"  constitutes  the  only  available  source 
of  personnel  capability  data. 

The  use  of  the  method  as  a  means  for  relating  system  requirements 
(standards)  and  real  world  potentialities  is  described  below. 

1.  Illustrative  Case 


To  illustrate  the  use  of  the  integrative  procedure,  three  hypothetical 
sets  of  probabilities  have  been  generated  and  are  presented  in  Table  1.  Each 
set  is  to  be  interpreted  as  applying  to  a  system  represented  by  three  functions 
(Fi,  F2  and  F3)  such  that  F^  is  composed  of  three  elements,  and  both  F^ 
and  F3  comprise  four  elements  each.  The  three  sets  of  values  are  totally 
independent  and  may  be  imagined  to  apply  to  three  different  systems,  or  as 
three  mutually  exclusive  hypothetical  possibilities  for  a  single  system.  In 
any  event,  the  non-parenthetical  numbers  represent  the  probabilities  with 
which  humans  can  be  expected  to  perform  the  associated  PEF  Units,  i.  e.  , 
reliable  human  capability  values.  The  numbers  in  parentheses  are  derived, 
allocated  probabilities,  weighted  on  the  associated  hypothetical  expectancy 
values;  those  numbers  represent  required  performance  levels. 

It  is  assumed  that  the  MSSM  for  each  Function  is  a  simple 
multiplicative  model;  thus,  the  product  of  the  capability  values  for  the 
elements  in  any  one  row  in  Table  1  is  equal  to  the  number  under  Sc  in  that 
row.  Similarly,  the  product  of  the  associated  parenthetical,  allocated 
values  in  a  single  row  equals  Sj..  the  operational  requirement.  In  most 
of  the  subsequent  discussion,  we  will  be  interested  in  comparing  system 
operational  requirements  (Sj.  values)  with  the  derived  capability  scores 
(Sc  values)  as  well  as  in  comparing  the  allocated  and  capability  values  at 
the  element  level.  The  kinds  of  considerations  and  operations  for  the 
comparisons  at  the  two  levels  are  quite  similar. 

The  t};ree  sets  of  hypothetical  data  (non-parenthetical  values) 
were  generated  so  that  (1)  in  Set  I,  the  Sr  values  exceed  Sc  values; 


Payne  and  Altman,  op.  cit.  ,  196^i 
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(2)  in  Set  II,  all  capability  scores  are  larger  than  the  corresponding 
required  values;  and  (3)  in  Set  III  there  is  one  high  probability  and 
two  Sc  values  that  are  below  Sr.  but  the  system  values  are  about  the 
same.  It  may  be  noted  that  the  probabilities  of  accomplishing  system 
output  states  are  obtained  by  multiplying  the  three  values  directly  above 
each  of  them;  for  example,  in  Set  I, 

System  Sj.  =  [.  940]  [.  939]  [.  951]  =  .  839  and 
System  Sc  =  [.  854]  (.  919]  [.  907]  =  .  712 

2.  Interpretation 


Discrepancies  between  corresponding  Sc  and  Sr  values  --  for  any 
set  of  data  --  may  result  from  one  or  more  of  the  following  conditions: 

a.  Human  capabilities  differ  from  those  required,  and 
changes  in  skill  levels  or  number  of  personnel  are 
indicated  through  modifications  in  training  or 
selection  procedures. 

b.  One  or  more  components  of  the  System  Effectiveness 
Requirement  (SER)  at  one  or  more  levels  of  specificity 
is  incompatible  with  that  which  can  be  achieved  for 

the  system  design  used  in  the  analysis. 

.  A  different  system  design  could  meet  the 
requirements . 

The  criteria  used  for  setting  the  SER  do  not 
correspond  to  that  dictated  by  system  context. 

c.  The  SER,  in  combination  with  decision  criteria,  is  set 
unrealistically  high  for  any  conceivable  system  design 
(or  too  low  for  the  system  to  be  useful). 

*  d.  Decision  criteria  used  in  the  allocation  process  are 
unnecessarily  stringent  or  does  not  correspond  to 
actual  system  context  requirements, 

e.  The  system  may  not  be  appropriately  modelled,  either 
graphically,  mathematically,  or  both. 

For  example.  Set  I  might  represent  a  system  whoso  design  may 
need  to  be  modified.  Excluding  lor  the  moment  che  possibility  of  other 
bases  for  the  discrepanc.es  betwt  en  S^  and  S^.  each  Function  would  need 
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Table  1 


Three  Hypothetical  Sets  of  Performance  Standards  and 
Corresponding  Capability  Levels  with  the  System  Operational 
Requirements  that  Must  be  Met 


Data 

Set 

Function 

1 

Element 

2  3 

4 

Capability 

Sc 

Required 

Sr 

Fl 

(.  993) 

.  983 

(.  988) 
.971 

(.958)1 
.  895  ^ 

---- 

.  854 

.  940 

I 

Fz 

(.984) 

.  978 

(.  987) 

.  983 

(.  979) 

.  970 

(.988) 

.  986 

.  919 

.  939 

F3 

(.  990) 

.  982 

(.  988) 

.  978 

(.  989) 

.  980 

(.983) 

.  964 

.  907 

.  951 

System 

.712 

.  839 

Fl 

(.957) 

.965 

(.988) 

.990 

(.  994) 

.  995 

.  951 

.  940 

II 

Fz 

(.983) 

.  ^98 

(.  973) 

.  997 

(.991) 

.  999 

(.991) 

.999 

.  993 

.  939 

F3 

(.993) 

.997 

(.  996) 

.  998 

(.  985) 

.  994 

(.976) 

.  991 

.  980 

.  951 

System 

.925 

.  839 

Fl 

(.  965) 

.  944 

{.  986) 

.  976 

(.  988) 

.  981 

— 

.  904 

o/jn 

III 

F? 

(.992) 

.  999 

(.'^83) 

.  998 

(  971) 

.  997 

(.992) 

.  999 

,  993 

.  939 

1 

1 

F3 

(.992) 

.  991 

(.961) 

.  975 

(.  990) 

.  986 

(.  987) 
.982 

936 

.  951 

System 

.  840 

.  839 

Parenthetical  numbers  are  derive*'’,  allocated  probabilities;  e.g.,  note 
that  (.  993)  (.988)  (.  958)  r  .  940  =  Sr  f.  'i  of  Set  I. 


Expected  probabilities  of  personnel  p  r*  mance  obtained  from 
capability  data. 
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iigned.  That  does  not  n*ean  that  the  probability  of  accomplishing 
each  element  within  a  given  function  must  be  increased  as  a  result  of  the 
redesign;  it  is  conceivable  that  in  producing  a  large  increase  in  the 
probabilities  of  accomplishing  some  elements,  others  may  be  forced  down 
slightly.  For  example,  the  probability  associated  with  Element  4  in  F2 
might  tolerate  a  reduction  of  .001  or  .002  ir.  order  to  increase  the 
probabilities  for  accomplishing  the  other  three  elements. 


Siixiilarly,  time  allowances  (where  critical)  and  tolerances,  or 
other  output  state  measures,  may  need  to  be  altered  by  redesign  so  that 
probabilities  of  accomplishment  are  increased.  As  long  as  the  system 
effectiveness  specifications  do  not  state  requirements  at  the  element 
level,  reconstruction  of  the  design  at  that  level  theoretically  can  assume 
any  form.  Of  course,  redesign  must  always  take  into  consideration  such 
additionally  specified  decision  criteria  as  personnel  training  requirements 
and  cost. 


It  is  also  conceivable  that  the  original  requirement  of  .  839  is 
higher  than  needed  in  the  system  context.  All  other  things  being  equal, 
if  the  system  could  actually  serve  its  purpose  by  supplying  a  specified 
output  with  a  probability  of  .  700  (Sr  =  •  700),  then  the  values  in  Set  I  are 
satisfactory  (Sc  =  •  712). 

Almost  any  effectiveness  requirement  whicn  is  possible  could  be 
met  if  cost  and  time  to  realize  the  design  were  unlimited.  However, 
restrictions  on  economy  of  all  kinds  normally  set  limits  on  what  may 
reasonably  be  expected  from  a  system.  Considering  the  immense  number 
of  permutations  and  combinations  of  elements  in  a  system,  it  is  not  a 
simple  matter  to  specify  a  general  procedure  for  determining  the  non¬ 
feasibility  of  achieving  system  requirements.  The  decision  may  have  to 
rest  in  the  hands  of  the  responsible  system  designers  who  are  unable  to 
find  alternative  means  of  pro  viding  the  required  system  ou1p”t  state.  As 
a  result,  either  the  effectiveness  requirements,  the  decision  criteria,  or 
both  may  need  to  be  relaxed  somewhat. 

Just  the  reverse  may  be  true  for  the  system  represented  by  the 
data  in  Set  II  of  Table  1,  where  Sr  is  less  than  S^-.  That  condition  could 
arise  if  the  effectiveness  requirement  and/or  decision  ciiteria  are  more 
relaxed  than  the  system  mission  actually  requires.  On  the  other  hand,  it 
may  well  be  true  that  the  system  mission  can  in  fact  be  accomplished 
successfully  at  the  originally  stated  requirement  level;  Sc  >Sr  n.ight  then 
be  the  result  of  overdesign.  An  overdesign  indicated  by  a  large 
discrepancy  between  and  Sr  generally  corresponds  to  an  expenditure  of 
hardware  or  personnel  (number  or  capabilities)  greater  than  necessary, 
i.e.,  excess  costs  in  general.  Note,  however,  that  it  may  be  possible  to 
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trace  overdesign  to  particular  elements.  For  example,  in  F^  of  Set  II, 
only  Element  1  may  be  overdesigred  while  Elements  Z  and  3  appear  to 
have  very  similar  allocated  and  capability  probabilitv  values.  Similarly, 
the  associated  values  in  Element  2  of  F3  are  almost  identical. 

When  applying  the  technique  to  an  existing  system,  it  is  in  the 
best  economic  interests  to  investigate  all  possible  sources  of  a  difference 
between  and  before  deciding  upon  a  course  of  action.  One  source 
which  has  not  been  mentioned  so  far  could  lie  not  in  the  system,  but  in 
the  basis  of  the  decisiot'  that  a  discrepancy  exists.  That  is,  the  system 
may  not  have  been  modeled  accurately,  so  that  an  erroneous  conclusion 
is  drawn.  It  is  likely  that  such  errors  will  occur  less  and  less  frequently 
as  experience  is  gained  with  the  use  of  the  technique.  However,  in  the 
initial  phases  of  its  application,  some  means  for  double -checking  the 
accuracy  of  the  models  may  be  necessary. 

Considering  Set  III  in  Table  1,  it  is  evident  that  if  no  Sj-  values 
were  specified  at  the  Function  level,  the  overall  system  effectiveness 
requirement  of  .  839  could  be  met  as  indicated  by  the  results  of 
integrative  analysis  (.  840),  However,  Function  requirements  are  indicated, 
and  for  two  of  the  Functions  (Fj  and  F3)  the  capability  scores  indicate 
inability  to  meet  those  requirements.  That  inability  is  particularly  reflected 
in  Elements  1  and  2  of  and  Elements  2,  3,  and  4  of  F3.  On  the  other 
hand,  the  ability  to  perform  F^  is  much  greater  than  specifications  demand. 

Conclusions  and  possible  actions  resulting  from  these  findings  are 
approached  in  the  same  way  as  has  been  discussed  above  where,  lor  the 
entire  system,  all  values  were  higher  or  lower  than  Sr  values.  However, 
when  some  are  higher  and  some  are  lower,  it  is  less  likely  that  the  source 
of  the  discrepancies  lies  in  erroneous  modeling,  unless  there  is  some 
reason  to  suspect  that  the  analyst  is  differentially  treating  seme  Functions 
with  greater  care  or  accuracy  than  others.  Also,  where  the  Function 
specifications  are  not  all  met,  but  system  effectiveness  requirements  are, 
the  added  alternative  exists  to  re-evaluate  the  basis  for  establishing  require¬ 
ments  at  the  Function  level.  It  may  be  less  costly  and  serve  the  mission 
purpose  to  accept  the  overall  system  as  it  stands,  rather  than  to  seek  a 
means  of  revising  Elemental  PEF  Units  to  meet  the  standards  at  the  Function 
level. 

G.  Overview  of  Procedure 


The  output  of  the  method  development  e  fort  described  in  the  preceding 
sections,  in  conjunction  with  method  application  on  the  AN/SPS-40,  resulted 
in  a  procedure  for  applying  the  method  to  establish  performance  standards. 
Detailed  step-by-step  procedures  are  presented  in  Appendix  B.  A  brief 
overview  of  the  procedure  in  graphic  model  form  is  shown  in  Figure  7  and 
is  summarized  below  by  procedural  steps. 
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1 .  Collect  Input  Data 


Obtain  system  operational  requirements  data,  system  descriptive 
data,  design  goals  and  data  concerning  functions  performed,  equipments 
involved  and  so  on.  Dat^.  sources  will  vary  dependmg  on  whether  the 
system  already  exists  or  is  in  the  conceptual  or  design  slate.  If  the 
system  is  an  existing  one,  sources  of  data  should  be  ~ystem  development 
and  test  reports,  operation  and  maintenance  guides  and  procedures, 
personnel/equipmeat  anaiysis  data  and  on  site  observations.  If  the  system 
is  in  the  design  stage,  sotirces  might  be  system  planning  and  research 
reports.  Tentative  Specific  Operational  Requirements  (TSOR),  Proposed 
Technical  Approaches  (PTA.'s),  Specific  Operational  Requirements  (SOR), 
Technical  Development  Plans  (TDP’s)  and  interviews  with  cognizant 
design  personnel.  (Data  collection  may  be  continued  until  the  GSSM  is 
completed. ) 

2,  Define  System  Input  and  Output  States 

Organize  the  data  and  define  all  relevant  features  of  the  input  and 
output  states  of  the  total  system  as  precisely  as  possible.  A.11  constraints 
or  assumptions  about  the  general  state  of  the  system,  as  for  example, 
state  of  alert,  should  also  be  specified  in  this  step. 

3.  Define  System  Functions 


On  the  basis  of  the  data  collected  and  the  input-output  states, 
define  the  general  Functions  mediating  the  input  and  output  states  of 
the  system. 

4.  Prepare  Graphic  Model  Data  Form  ^CMDF) 


Prepare  detailed  GMDF's  for  each  Fencliou  detailed,  one  for 
each  level  of  specificity.  The  GMDF  should  include  complerc  specificatn 
of  all  meaningful  input  and  output  states  frr  every  PEF  Unit. 

5 .  Construct  Graphic  State  Sequence  Model  (GSSM) 


Based  on  the  GMDF's  and  the  GSSM  construction  guidelines, 
develop  s^ate  sequence  models  for  each  relevant  level  of  each  function 
and  illustrate  graphically  the  relations  among  all  PEF  Units  with  all 
possible  meaningful  input  and  output  states. 
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Cons truct  Mathematic  State  Sequence  Model  (MSSM) 


From  the  GSSM  configuration  and  the  principles  of  proba.bility 
theory,  specify  the  probability  of  achieving  the  system  output  state  as  a 
function  of  the  probabilities  of  accomplishing  the  personnel  equipment 
amctions  within  the  particular  system  context. 

7,  Allocate  Require  nents  to  PEF  Units 


Determine  the  independently  derived  Index  of  Task  Accomplish¬ 
ment  (la)  for  each  PEF  Unit.  From  the  probability  function  of  the  la  value 
and  the  MSSM,  compute  the  probabilities  of  success  of  the  PEF  Unit 
output  states  (standards)  necessary  for  total  system  requirements  to  be 
met.  Repeat  the  allocation  determination  for  extreme  alternatives,  and 
determine  the  possible  range  of  PEF  Unit  output  state  probabilities 
which  will  satisfy  the  total  system  output  requirement. 
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TV.  EXTENSION  OF  BASIC  CAPABILITY 


In  Section  I-B  (Scope),  it  was  noied  that  the  present  study  v/as  devoted 
to  developing  a  basic  capability  for  meeting  two  methodological  objectives. 

This  section  considers  additional  problem  dimensions  and  represents  a 
first  step  in  extending  that  capability. 

A,  Integration  of  Maintenance  Considerations 

While  the  description  of  method  development  in  preceding  sectioxis  has 
been  oriented  toward  developing  standards  for  operator  performance,  the 
same  general  procedures  and  modeling  approach  are  applicable  to  system 
maintenance.  The  basic  distinctions  are  that  for  maintainer  performance: 

(})  time,  rather  than  probability  of  success,  is  the  primary  standard  of 
performance;  and  (2)  due  to  the  fact  that  the  probability  of  failure  need  only 
be  introduced  at  the  most  general  level  of  modeling,  it  is  not  necessary  to 
include  failure  and  maintenance  considerations  in  the  detailed  graphic  and 
mathematical  state  sequence  models  used  to  represent  operator  performance. 

Overall  corrective  maintenance  requirements  are  derived  from  availability 
or  dependability  requirements  established  for  the  system  or  subsystem. 
Availability,  A,  or  the  expected  probability  that  the  system  will  be  operating 
at  a  specified  point  in  time  can  be  expressed: 

A  =  MTBF/(MTBF  +  MTTR>, 

where  MTBP'is  Mean  Time  Between  Failure  and  MTTR  is  Mean  Time  To 
Restore  the  system  or  equipment  to  operation  in  the  event  of  failure.  MTTR 
thus  constitutes  the  most  general  standard  for  maintenance  performance. 

Since  a  system  faiiur!?,  potentially  constitutes  the  output  state  of  any 
personnel/equipment  functional  unit,  the  occurrence  of  a  maintenance  task 
can  be  incorporated  in  the  system  state  sequence  model  as  illustrated  below: 
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The  maintenance  task  can. 


■J  f 


be  modeled  in  more  detail  as  shown: 


(System  ^ 

Fault 

Fault 

N 

- 1 

Fault 

i 

N 

Fault 

Fault 

(System 

failed) 

Detection 

Localization 

1 

Isolation 

1, 

Correction 

Verification 

restored)' 

The  probability  of  successfully  completing  the  maintenance  task  is,  by 
definition,  unity.  The  probabilities  of  completing  the  maintenance  subtasks 
illustrated  above,  howevex*,  are  not  necessarily  equal  to  unity  and  may 
therefore  affect  overall  task  time  (e.g.  ,  the  replacement  of  a  good  part 
incorrectly  identified  as  being  failed  increases  total  task  time).  Since 
statistics  pertaining  to  the  probability  of  performing  maintenance  subtasks 
are  not  necessarily  required  in  the  development  of  standards,  primary 
consideration  will  be  given  to  time  parameters  of  tiie  subtasks. 

At  the  extremes  of  specificity  of  standards  for  the  maintenance  task  are 
the  overall  MTTR  requirement,  and  standards  to  perform  each  maintenance 
task  which  might  be  required  in  the  system.  The  latter  standards  car.  be 
determined  theoretically  using  the  fixed  relation  between  (1)  the  failure  rates 
and  the  required  maintenance  times  for  all  elements  in  the  system  and  (2)  a 
basis  for  time  allocation  such  as  an  Index  of  Task  Accomplishment.  The 
relation  between  system  MTTR  and  individual  maintenance  actions  may  be 
expressed  as  follows: 

n  n 

MTTR  =  (  2  Xj  R;)  /  2  \- 
i=l  ^  ^  i~l  ^ 

where: 

R^  =  time  to  carry  out  maintenance  on  the 
i  th  element 

Xj  =  failure  rate  of  the  i  th  element 

n  =  number  of  possible  maintenance  actions 
in  the  system  (equals  the  number  of  on¬ 
line  replacement  elements) 

Since  most  systems  involve  many  maintenance  tasks  which  are  quite 
similar,  the  most  appropriate  approach  appears  to  be  to  develop  a  set  of 
standards  which  is  intermediate  in  specificity.  Depending  on  the  nature  of 
the  system  being  analyzed,  and  the  particular  method  in  which  the  standards 
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are  to  be  utilized,  maintenance  time  standards  could  .'e  established  for 
various  categories  of  maintenance  tasks.  An  illustrative  classification 
system  would  consist  of  the  following; 

1.  /jLaintenance  tasks  involving  automatic  localization  of  failed 
modules. 

2.  Maintenance  tasks  involving  the  use  of  test  equipment  to 
observe  simple  input-output  relations  to  isolate  failed  modules 
and  parts. 

3.  Maintenance  tasks  involving  more  complex  diagnostic  activities 
than  those  described  in  Categories  1  and  2. 

1  types  of  data  required  to  develop  standards  would  include  the 
following: 


•  Identification  of  the  maintenance  task  for  each  element^ 

(or  group  of  elements  maintained  in  a  similar  manner) 
in  the  system; 

"  The  category  of  maintenance  in  terms  of  human  capability 
requirements;  and 

•  The  failure  rate  of  each  element  (or  group  of  elements 
maintained  in  a  similar  manner). 

Carrying  out  the  procedures  outlined  in  MIL.-M-2331  3A  for  predicting 
mean  time  to  restore  would  satisfy  the  first  and  third  requirements;^  however, 
additional  characterization  of  the  various  maintenance  tasks  with  respect  to 
demands  on  human  capabilities  is  still  required.  In  the  development  of 
standards,  failure  and  maintenance  considerations  need  only  be  considered  a^ 
the  most  gross  level;  it  is  not  necessary  to  reproduce  those  considerations  in 
the  detailed  graphic  or  mathematical  models.  The  major  task  involved  in 
establishing  standards,  therefore,  consists  in  using  the  data  above  plus  an 
Index  of  Task  Accomplishment,  simila*  to  that  described  in  Section  III-Eto 
allocate  standards. 


In  this  section,  element  refers  to  the  hardware  item  on  which  the  mainte¬ 
nance  action  takes  place,  i,  e,  ,  module,  piece  part,  etc. 

Maintainability  Requirement  for  Shipboard  and  Shore  Electronic  Equipment 
and  Systems,  MIL-M-2331  3A,  9  October  1963. 
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To  illustrate  the  approach,  it  is  assumed  that  the  combined  failure  rates 
of  elements  maintained  according  to  the  three  categories  described  above 
have  been  determined;: 


Maintenance 

Category 


Failure  Rate  of 
Elements  in  the  Category 
(failures  per  hour) 


1  .  002 

2  .  006 

3  .  002 

Total  System  .  010 


Substituting  known  values  in  the  MTTR  relation  given  above,  one  obtains: 
MTTR  =  0.  2Rj  +  0.  bR^  +  0.  2R3 

If  it  is  assumed  that  the  Indices  of  Task  Accomplishment  (1^  's)  are  1, 

2,  and  3  for  maintenance  categories  1,  2,  and  3,  respectively,  the  following 
relation  can  be  stated: 


and  substituting  in  the  MTTR  relation,  assuming  the  overall  MTTR  is  1. 0 
hour, 


MTTR  =  1.0  =  0.  2Ri  +  1.  2Rj  +  0.  6Rj 

1.0=  2Rj  ;  Rj  =  0.  5  hours 

R^  =  1.0  hours 
R3  =  1.5  hours 

The  complete  set  of  standards  for  system  maintenance  would  then  be; 
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maintenance 

Category 

fciiiure  Kate 
(failures  per  hour) 

Time  Requirement 
(hours) 

1 

.  002 

0.  5 

2 

.  006 

1. 0 

3 

.  Oi  iL 

1.  5 

Total  System 

.  010 

1.  0 

More  detailed  maintenance  task  classifications  could  be  established  for 
Ary  specific  system.  The  selection  cf  an  exact  classification  system  would 
take  into  account  the  variety  of  maintenance  tasks  in  the  system  as  well  as 
training  and  occupational  hierarchy  considerations.  Additionally,  techniques 
may  be  developed  for  establishing  standards  for  maintenance  subtasks. 


B.  Consideration  of  Additional  Standards  Components 

Means  of  determining  tinie  and  accuracy  components  of  PEF  Unit 
standards  are  not  included  in  the  basic  capability  represented  by  the  method 
described  in  Section  III.  If  those  components  of  performance  standar'^c  are 
taken  into  explicit  consideration  and  not  held  constant,  the  allocation 
procedure  becomes  increasingly  complex  since  it  is  not  likely  that  cc  po- 
nents  are  independent.  The  solution  of  the  Mathematical  State  Seque-je 
Model  will  yield  different  results  as  probability,  time  and/or  accuracy 
values  are  varied  for  one  or  more  PEF  Units.  Efforts  to  trade-off  time 
and  accuracy  or  time  and  probability,  for  example,  will  resrlt  in  simultaneous 
changes  in  allocated  probability,  time,  and  accuracy  requirements  throughout 
the  system.  Thus,  the  interdependence  of  all  components  must  be  appropri¬ 
ately  reflected  by  the  technique.  This  section  explores  various  approaches 
for  integrating  time  and  accuracy  components  with  the  probability  of 
accomplishment  component. 

1 .  Preliminary  Considerations  and  Assumptions 

The  exact  nature  of  the  Index  of  Task  Accomplishment  (IOTA)  still 
remains  to  be  specified,  although  in  Section  IIl-E  it  was  suggested  that  it 
might  be  a  derived  "score  "  or  a  value  which  is  a  function  of  several  weighted 
variables.  While  time  and  accuracy  were  not  included  in  the  previous 
discussion,  a  more  complete  IOTA  function  could  be  described  which  varies 
with: 
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•  Man's  capability,  M, 

•  Difficulty  of  the  job,  J, 

•  Tinrie  allowed,  t,  and 

•  Accuracy  required  (measurement  tolerance  permitted),  L. 
If  i  is  defined  as  the  variable  IOTA, 

i  =  flM,  J,  t,  L).  (1) 

It  is  assumed  that  M  and  J  can  be  defined  as  discrete  variables,  ^ 
while  t  and  L  are  continuous  such  that: 

—  >  0;  ^  >0 
at  -  8L  - 

i(M  +  c,  J,  t,  L)  -  i(M,  J,  t,  L)  >  0 
L(M,  J  +  €,  t,  L)  -  i(M,  J,  t,  L)  <0 
where  e  is  a  small  positive  increment. 

The  relation  above  indicates  that  i  reflects  the  "ease"  with  which  a  man  can 
perform  a  job,  sL.ce  i  increases  as  (1)  more  time  is  allowed,  (2)  more 
tolerance  is  allowed  (accuracy  requirement  relaxed),  (3)  man's  capability 
increases,  and  (4)  job  difficulty  is  reduced. 

In  order  to  treat  the  time  and  accuracy  components  separately, 
Equation  (1)  may  be  rewritten: 

L  =  f{fi  (M,  J),  f,(t),  f3(L)}  (2) 

In  the  subsequent  discussion,  the  probable  forms  of  functions  £2  and  f^  in 
Equation  (2)  will  be  discussed. 


Time  and  accuracy  variables  can  be  treated  as  discrete;  however,  such 
an  approach  would  require  the  development  of  a  large  number  of  matrices 
relating  time  and  accur.icy  for  each  probability  level  and  for  every  PEF 
Unit. 
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2.  An  Example  Relating  the  Index  of  Accomplishment  and  Tirnt? 


Onp  basic  assumption  concerning  the  time  variable  is  that  a  maximum 
value  exists  for  time,  beyond  v-hich  i  does  not  change,  if  the  accuracy 
requirement  is  held  constant.  That  value  o.^’  i  is 

.  =  h  =  fhl  (M,  J)f2(T^3^)f3  (L)} 

lim  t  ->eo 

(L  constant)  =  f{fj  (M,  J)  f,  (L)}  (3)^ 

To  aid  in  the  cerivation  of  iA  t)  and  the  logic  behind  the  above  assumption, 
a  hypothetical  example  will  be  used.  Consider  that  100  people  of  relatively 
equal  capability  are  asked  to  perform  a  certain  task  as  rapidly  as  possible; 
the  task  is  a  go/no-go  type  which  does  not  involve  an  accuracy  level..  We 
would  expect  the  results  --  number  of  people  versus  time  to  perform  the 
task  --  to  be  normally  or  lognormally  distributed  over  time.  ^ 

Of  the  100  people,  some  will  not  complete  the  task  properly  even 
though  they  may  have  thought  they  did.  Let  us  say  that  95  of  them  succeeded. 

It  will  be  assumed  then,  that  the  times  for  successful  and  unsuccessful  persons 
are  both  distributed  in  the  same  way.  It  will  also  be  assumed  that  if  the  same 
group  had  all  the  time  they  needed  to  do  the  same  task,  the  probability  of  a 
successful  performance  would  still  be  .  95.  This  assumption  needs  to  be 
tested,  but  reflects  the  consideration  that  in  spite  cf  the  advantage  gained  by 
removing  the  time  constraint,  we  might  expect  degradation  ’n  performance 
due  to  other  factors.  For  example,  those  who  customarily  work  fast  might 
lose  interest  and  become  careless  if  they  were  not  expected  to  use  their 
speed  skills.  Also,  personnel  under  observation  usually  make  a  conscious 
effort  to  display  better  performance  than  under  normal  conditions,  so  that 
a  distorted  picture  might  be  attained  unless  the  people  did  not  know  they  were 
being  obser'-ed.  But  even  if  personnel  had  all  the  time  they  needed  in  an 
actual  operational  situation,  there  is  some  question  as  to  whether  they  would 
respond  to  temporal  freedom  constructively.  These  considerations  could 
serve  to  cancel  out  any  advantage  gained  by  placing  no  time  limit  on  task 
performance. 


It  may  be  noted  that  Equation  (3)  defines  I^^  as  it  is  used  in  Section  III-E  and 
in  Appendi.x  B. 

See  (I)  Conrad,  R  and  Hille,  B.  A,  'Comparison  of  Paced  and  Unpaced 
Performance  at  a  Packing  Task,”  Occup.  Psych.  ,  1955,  ^  15-28,  and 

(2)  Harrison,  G.  ,  et  al.  Maintainability  Prediction:  Theoretical  Basis 
and  Practical  Approach  (Revised).  Washington,  D.  C.  :  A.RIKC  Research 
Corp,  Publication  267-02-6-420,  December  1963. 


3.  A  Proposed  Time  Function 


Based  on  the  two  assumptions  above,  it  should  be  possible  to  plot 
i  vs  t  assuming,  as  stated  earlier,  that  the  ‘Originally  computed  for  a 
given  task  is  the  index  of  accomplishment  when  no  time  limit  is  imposed. 

In  the  example  above,  that  would  mean  that  is  associated  with  the  .  95 
proportion  of  successful  personnel,  and  i  would  be  expected  to  be  a  monotonic 
function  of  time,  related  lo  the  integral  of  the  normal  or  lognormal  distribution. 


We  will  proceed  by  assuming  a  task  whose  observed  frequency 
distribution  of  accomplishment  best  fits  a  Gaussian  (normal)  curve.  Under 
that  assumption,  it  would  be  expected  that  i  might  closely  fit  the  following 
function: 


1 


exp. 


(T-rr 

Zoy.  J 


dr 


(4) 


where 

r  =  the  independent  variable  (time)  of  the  Gaussian 
distribution  fitted  to  the  observed  data 

r  =  the  mean  time  foi  that  distribution 

oy  =  the  standard  deviation  of  the  distribution 

t  =  integration  time 

exp.  -  exponent  of  e;  i.  e.  ,  (exp.  x)  = 

Equation  (4)  fails  to  model  all  aspects  of  the  expected  distribution, 
however,  because  it  has  a  non-zero  value  for  t=0,  and  i  would  be  expected 
to  be  zero  below  a  minimum  positive  time,  g-  ;  would  not  expect 

to  find  anyone  who  can  tune  a  receiver  in  a  fev/  milliseconds,  if  it  is  not 
already  tuned).  Additionally,  it  seems  a  reasonable  a.;sumption  that  beyond 
some  maximum  time,  i  will  remain  constant,  i.  e.  ,  beyond  Tj^a,x»  ^  ~  ^a> 
however,  Equation  (4)  would  specify  that  Txna.x  which  is  not  a  useful 

concept.  In  an  effort  to  incorporate  these  practical  constraints,  a  sinusoidal 
approximation  to  the  normal  curve  was  developed.  Subsequently,  it  was 
discovered  that  Raab  and  Green^  suggested  a  similar  approximation  which  is 


Raab,  D.  H.  and  Green,  E.  H.  "A  Cosine  Approximation  tc  the  Normal 
Distribution,  "  Psychometrika,  26,  447-450  (1961). 
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nJightly  simpler  than  the  one  devived.  to  we  elected  lo  use  theirs;  its 
general  form  is 


f(x) 


I 

TT' 


(1  +CGS  S.) 


for  -iT<x<Tr  (5) 


Utilizing  our  symbology  and  assumptions,  then,  Equation  (4)  can  be 
approximate  U: 


(7) 

(8) 


Solving  Equation  (6),  it  can  be  shown  that 


■  ^min  1 

1  =  E  I -  + 

^1  2  IT  ov  ”21i 


t  -  T. 


f“’'  Tmax>  *  >  Tmin  <9) 


Additionally,  Equations  (7)  and  (8)  still  apply  when  t  is  substituted  for  T. 

Referring  back  to  Equations  (2)  and  (3),  it  may  be  noted  that  the 
bracketed  portion  of  Equation  (9)  is  the  proposed  f2(t). 

In  essence,  it  appears  likely  that  data  obtained  from  observing  task 
perforrr  ance  will  display  a  normal  (or  lognormal)^  distribution  of  frequency 
of  task  accomplishment  as  a  function  of  time.  It  is  also  likely  that  the  means 

If  a  lognormal  distribution  is  found,  i.  e.  ,  if  the  distribution  is  highly  skewed 
toward  longer  time  values,  logT,  log  and  log  must  be  substituted 

forr,  and  respectively,  in  Equations  (6),  (7),  and  (8). 
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(f)  and  standard  deviations  (o'y  )  will  dificr  for  different  t-iisks,  so  that  the 
interval  represented  by  *  Tmin  will  vary,  depending  upon  the  "com¬ 

plexity"  of  the  task.  The  data  may  also  show  other  regular  relations,  such 
as  a  correlation  between  T  and  Cy-  ;  it  would  not  be  unreasonable  to  expect 
that,  as  task  "difficulty"  increases,  both  the  mean  tine  to  accomplish  a  task 
and  the  variability  would  also  increase  simultaneously.  \/hether  the  relation 
is  definable  and  reliable  remains  to  be  tested. 

Detertrining  the  Index  Value  for  Allocation 

If  no  time  limit  is  specified  for  a  system,  the  PEF  Units'  index  of 
accomplishment  is  its  maximum  i  value,  or  When  quantities  have  been 
calculated  for  all  PEF  Units,  those  maximum  indices  become  the  values  which 
are  scaled  so  as  to  achieve  thv^  system  effectiveness  requirement,  F^,  via  the 
MSSM.  It  is  assumed  that  scaling  values,  according  to  the  hypothetical 
monotonic  transformation  discussed  in  the  allocatic-n  section,  satisfies  the 
secondary  criteria  of  minimizing  cost  or  training  or  some  com.bina’^ion  of 
factors.  Thus,  it  has  also  been  assumed  tnat  the  i  function  depends  solely 
upon  1-^  and  time  and  that  its  variation  in  time  does  not  affect  the  satisfaction 
of  the  secondary  criteria. 

If  a  time  limit  i^  specified  for  the  system,  the  effectivenes.s 
requirements  include  Tj,  the  total  allowable  time  to  achieve  the  output  state, 
as  well  as  P^,  the  probability  specified  for*  the  system  output.  As  a  result, 
all  MSSM-related  PEF  Unit  probabilities  of  accomplishment  must  be 
calculated  so  as  to  satisfy  both  requirements.  Combining  the  probability  and 
tim^  functions  to  solve  the  MSSM,  then,  assumes  not  only  that  there  is  a 
definable  function,  f2(f)*  lo*"  every  PEF  Unit,  but  also  that  i  values  for 
different  PEF  Units  (1)  are  comparable  (i.  e.  ,  lie  on  the  same  scale)  sc  that 
the  values  can  be  weighted  relatively  to  satisfy  without  altering  their 
interrelations,  and  (2)  are  independent,  sc  that  no  error  is  generated  by 
bringing  them  together  in  the  MSSM. 

To  maximize  the  accomplishment  probabilities  and  simultaneously 
meet  the  system  effectiveness  requirements  involves  finding  the  best  solution 
to  a  series  of  simultaneous  equations.  To  iilustrace  the  processes,  a  simple 
hypothetical  example  will  be  used. 

Assume  a  system  composed  of  three  PEF  Units  such  that  the  MSSM 
is 

Po=  J,  P(Oa/y  (10) 

where  is  the  input  state  and  is  the  output  state  of  the  ath  PEF  Unit. 
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f  rom  previous  assuroptions  regarding  the  monotonic  relation  between  IOTA 
and  the  probability  of  task  accornplishinai t,  the  following  functional  relation 
has  been  established; 


P(Oa/I^)  =  f(i(i)  (11) 

for  all  three  a.  Also,  from  Equation  (9),  we  can  indicate  generally  that 

■a  =g(tj  (12) 

where  g  is  a  different  function  from  the  f  in  equation  (11).  This  assumes 
that  for  each  PEF  Unit,  and  a  are  known. 

Substituting  in  Equation  ilO),  f.quation  (13)  results  for  a  specified  Pq. 
3 

Po=  f{g(t<^}  (13) 

a=l 


Equation  (13)  is  constrained  by  the  requirement  that 
3 

E  tjj  ^  T  T  ( 1  '^) 

u-T 

It  is  possible  that  there  is  more  than  one  solution  to  Equation  (13), 
even  under  the  constraint  of  Equation  (14),  since  f(ia  )  involves  an  unknown 
constant-  k.  However,  additional  analysis  is  required  to  establish  the 
appropriate  criterion  for  specifying  the  unique  solution.  Sutfice  it  here  tc 
say  that  one  may  reasonably  expe-;t  the  time  component  of  the  standard  to  be 
determinable  mathematically,  if  the  basic  assumptions  --  which  have  been 
veroalized  throughout  this  section  -  -  are  tenable. 


5 .  Determining  th e  Accuracy  Component  of  the  Standard 

In  a  similar  way,  f^  -  the  function  of  the  accuracy  comp*"  f^t  of  the 
standard  --  in  Equ.ition  (2)  may  be  estimated.  This  amounts  to  deteimining 
the  effects  of  various  accuracy  requirements  associated  with  output  state 
measures.  Assuming  time  to  be  held  constant,  it  appears  likely  that  if  many 
people  were  given  the  task  to  measure  a  parameter,  the  results  would  be 
expected  to  be  distributed  normally  about  the  true  measure  of  that  parameter. 
Therefore,  using  Equation  (5)  as  the  approximation  to  the  Gaussian  curve, 
tlie  measurement  function,  f(m),  would  be 
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where  m  is  the  true  measure  and  is  the  standard  deviation  of  the  distri¬ 
buted  measures,  m.  Since,  however,  we  are  only  interested  in  determining 
measurability  v/ithin  a  tolerance,  say  iL,  then  the  distribution  can  be  looked 
upon  as  an  error  curve  with  ifi  =  O.  Therefore,  Equation  (1  5)  can  be  rewritten 


f(m') 


2  -tro, 
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where  m'  =  m-m. 


for  -IT  <  -  <  TT  (16) 

'^m 


To  determine  the  probability  of  measuring  within  ±  L,  Equation 
(1  5)  is  integrated  from  zero  to  L.  and  multiplied  by  two,  because  the  tolerance 
encompasses  the  center  of  the  Gaussian  distribution  symmetrically.  Thus, 
that  probability  is  approximated  by 


P(L)  = 


TTO- 


m 


dm’ 


P(L) 


L  < 


TT  01 


m 


(17) 


Equation  (17)  seems  to  correspond  to  what  might  logically  be 
expected,  viz.  ,  (1)  the  probability  of  measuring  a  value  increases  as  the 
allowable  tolerance  increases  and  (2)  the  probability  decreases  as  the  variance 
of  the  measures  increases.  However,  if  Equation  (17)  does  reliably  represent 
the  actual  relation,  it  still  remains  necessary  to  determine  or  estimate  0-^^. 
This  may  be  accomplished  b  .cllecting  controlled  observational  data  or  by 
obtaining  judgments  of  P(L),  given  selected  values  of  L,  by  a  number  of 
knowledgeable  judges.  It  is  possible  that  a  discrete  number  of  practicable 
estimates  of  05^  can  be  found  for  specific  classes  of  performance,  so  that 
P(L)  would  be  a  continuous  function  of  L  for  a  given  task  within  a  particular 
known  classification. 
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In  the  derivation  of  Equation  (9),  a  constant  ''alue  of  was  assumed. 
As  a  result,  it  is  evident  from  Equation  (3)  that  that  was  tantamount  to 
assuming  constant  values  for  M,  J,  and  E.  To  vary  L  m«-ans,  then,  that  Ig 
in  Equation  (9)  must  be  replaced  by  its  equivalent  fj(M,  J)  f-(L).  As  before, 
M  and  J  are  still  constant,  since  the  discussion  concerns  a  particular  set  of 
human  capabilities  operating  on  a  particular  job.  Therefore,  from  Equation 
(17)  it  seems  reasonable  to  define  f3(L)  =  P{L). 

To  conclude,  therefore,  it  is  not  much  of  a  problem  to  hypothesize 
{2  and  f^  of  Equation  (2),  but  it  seems  evident,  from  the  discussion  throughout 
this  section, that  the  real  problem  now  lies  in  obtaining  sufficient  pertinent, 
accurate  and  internally  consistent  date  to  support  or  refute  the  expected 
relations  which  hav'e  been  hypothesized  here. 
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V.  METHOD  APPLICATION 


The  job  standards  derivation  method  was  applied  to  an  operational 
Navy  shipboard  subsystem  for  the  purpose  of  testing  and  refining  the 
various  techniques  and  procedures  being  developed.  By  testing  the  method 
empirically  during  its  development  using  actual  system  data,  there  is 
added  assurance  that  tlie  method  possesses  sensitivity  and  practical  utility 
necessary  for  its  use  by  the  Navy  as  a  general  system  analysis  tool. 

Only  a  brief  discussion  of  method  applicatic>n  is  presented  here  due  to 
the  classified  nature  of  the  AN/SPS-40  system  data  analyzed.  The  numerical 
standards  established  for  the  various  AN/SPS-40  PEF  Units  are  presented 
in  Volume  II,  together  with  all  graphic  and  mathematical  state  sequence, 
models  and  an  appradsal  of  many  of  the  detailed  aspects  of  the  method 
application  to  the  AN/SPS-40. 

A.  Selection  of  Test  Subsystem 

Selection  of  a  test  subsystem  was  based  on  a  comparison  of  the  relative 
merits  of  several  AAW  and  ASW  subsystems.  The  selection  process 
proceeded  according  to  the  following  steps: 

.  Development  of  selection  criteria; 

.  Identification  of  AAW  and  ASW  subsystems; 

.  Data  collection  from  Navy  fleet  and  training  facilities; 

.  Relative  evaluation  of  subsystems  based  on  selection 
criteria;  and 

.  Joint  conference  with  contract  monitor  to  select  the  final 
test  subsystem. 

Selection  criteria  were  developed  according  to  two  major  guidelines: 

(1)  the  tryout  subsystem  must  have  the  characteristics  necessary  to  assure 
a  valid  test  in  relation  to  present  and  future  Navy  subsystems;  and  (2)  data 
procurement  and  analysis  time  must  be  consistent  with  the  project' s  time 
frame.  The  more  important  criteria  dealt  with  availability  of  effectiveness 
requirements,  documentation  on  opex*ating  and  maintenance  requirements 
and  the  functional  complexity  and  representativeness  of  a  subsystem  relative 
to  other  current  and  proposed  Navy  subsystems. 

The  original  selection  group  was  composed  of  29  Navy  subsystems 
representing  air  search  radars,  surface  sonars,  sub-surface  sonars  and 
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fire  control  subsystems.  After  several  evaluations  based  on  the  selection 
criteria,  the  AN/SPS-30,  AN/SPS-40  and  AN/SPS-43  air  search  radar 
subsystems  were  found  to  be  equally  satisfactory.  In  conjunction  with  Navy 
BuPers  representatives,  the  AN/SPS-40  was  finailly  chosen  as  the  most 
feasible  and  potentially  useful  subsystem  for  use  in  developing  and  testing 
the  standards  derivation  method. 

B.  Summary  of  Procedure  Employed 

The  procedural  format,  developed  in  part  during  the  trial  application 
and  used  to  establish  standards  for  the  AN/SPS-40,  is  presented  in 
Appendix  B  and  consists  of  the  following  steps: 

.  Collect  Data 

.  Define  System  Input  a. '.d  Output  States 

.  Determine  System  Functions 

.  Prepare  Graphic  Model  Data  Form  (GMDF) 

.  Construct  Graphic  State  Sequence  Model  (GSSM) 

.  Construct  Mathematical  State  Sequence-  Model  (MSSM) 

.  Allocate  Requirements  to  PEF  Units 

C.  Appraisal 

Results  of  method  application  warrant  a  highly  optimistic  outlook, 
although  there  are  some  theoretical  deficiencies  that  have  yet  to  be  over¬ 
come,  particularly  in  the  area  of  requirements  allocation. 

Much  system  functional  and  structural  data  were  needed  to  construct 
the  Graphic  State  Sequence  Model,  There  were  some  difficulties  in  locating 
all  the  necessary  documentation  required  because  system  effectiveness 
requirements  for  the  AN/SPS-40  have  been  defined  only  in  a  gross  sense. 

A  minor  difficulty  arose  in  setting  limits  (i.  e.  ,  stating  the  appropriate  input 
and  output  states)  to  that  portion  of  the  radar  subsystem  which  we  felt  could 
be  hcindled  in  the  time  available  and  at  the  same  time  provide  a  good  challenge 
to  the  methodology. 

Constructing  the  GSSM  definitely  requires  training  in  concept  and 
procedure.  However,  we  have  demonstrated  to  our  own  satisfaction  that 
such  training  can  be  accomplished  economically.  In  the  course  of  studying 
and  applying  instructional  procedures,  notes  were  kept  concerning  the 
important  issues  involved  in  communicating  principles  of  GSSM  construction. 
Those  notes  in  turn  were  refined  and  organized  into  the  systematic  format 
presented  in  Appendix  B. 
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Of  particular  importance  in  GSSM  structuring  is  the  need  for  a  search¬ 
ing,  open-minded  attitude  combined  with  an  understanding  of  what  all 
aspects  of  the  system  can  do.  System  design  must  not  restrict  the  analyst' s 
thinking  as  to  how  the  system  is  purported  or  expected  to  perform. 

Instead,  the  essential  question  must  concern  what  the  system  is  capable  of 
doing  under  various  circumstances.  Once  the  GSSM  was  constructed,  the 
mathematical  modeling  technique  was  applied  without  difficulties. 

Human  reliability  information  from  the  "data  store"  developed  by  Payne 
and  Altman^  was  used  to  "score"  the  AN/SPS-40  PEF  Units  and  therefore 
provide  a  basis  for  requirements  allocation.  The  limitations  of  using  that 
approach  have  been  described  in  Section  III-E. 

Aspects  of  allocation  not  considered  in  the  application  were:  (1)  treat¬ 
ment  of  alternative  procedures  for  arri  .ng  at  a  given  system  state;  and 
(2)  requirements  other  than  probability  of  accomplishment  (e.  g.  ,  performance 
time  or  accuracy).  The  first  aspect  resulted  from  the  large  number  of 
alternative  procedures  which  could  exist  within  the  AN/SPS-40  that  will 
lead  to  the  same  (or  a  highly  similar)  system  state.  Since  the  principal 
purpose  of  the  application  was  to  develop  and  test  the  method  rather  than  to 
perform  a  complete  analysis  of  the  AN/SPS-40,  it  was  decided  that  the  purpose 
could  be  served  most  efficiently  and  effectively  by  considering  only  the 
system' s  principal  activity.  The  process  for  allocating  system  require¬ 
ments  among  the  alternatives  would,  in  any  case,  follow  the  methods  detailed 
in  Section  III-B  and  Appendix  B. 

A  test  of  requirements  allocation  in  terms  other  than  probability  of 
accomplishment  was  not  carried  out  since  the  scope  of  the  study  did  not 
include  complete  development  of  multi-dimensional  allocation  techniques. 

It  is  felt  that  the  application  adequately  tested  the  method  through  attention 
to  the  effectiveness  requirement  with  the  greatest  potential  utility  -- 
probability  of  task  accomplishmerl.  In  light  of  the  successful  application 
of  the  method  to  the  AN/SPS-40,  there  is  every  reason  to  expect  that  the 
technique  can  be  appUed  successfully  to  a  wide  range  of  Navy  systems  for 
which  adequate  input  data  are  vobtainable.  Complete  and  currently  accurate 
inlormation  regarding  a  given  system  is  necessary  to  apply  the  graphic  and 
mathematical  models. 

The  approach  to  method  development  included  constant  consideration  of 
the  anticipated  needs  of  the  person  who  is  to  apply  the  technique.  As  a 
result,  any  capable  and  interested  person  should  be  able  to  learn  to  use 
the  technique  once  it  has  been  fully  developed.  In  its  present  state  of 
development,  the  modeling  tasks  have  been  defined  in  detail  and  should 
present  no  problem  in  application.  However,  due  to  the  absence  of  a 
compre’-'ensive  set  of  behavioral  data,  and  a  fully  developed  allocation 
procedure,  the  allocation  process  cannot  be  considered  a  routine  task. 

Payne  and  Altman,  op.  cit.  ,  1962. 
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VI.  POTENTL^L  USES  OF  THE  METHOD 


The  job  standards  method,  following  comprehensive  methodological 
development  and  testing,  can  be  expected  to  have  application  as  a  general 
research  tool  in  system  design  and  development  in  addition  to  its  perhaps 
more  obvious  and  immediate  role  in  personnel  research.  Potential 
applications  in  those  two  areas  are  described  below. 

A ,  Personnel  Research  and  Management 


Application  of  the  job  standards  method  provides  a  body  of  performance 
standards  or  criteria  tnat  reflect  actual  performance  requirements  of  the 
system  for  each  identifiable  peraonnel/equipment  functional  unit.  Con¬ 
sequently,  the  method's  potential  in  the  area  of  personnel  research  and 
management  (selection,  assignment,  organization,  training  and  evaluation) 
is  extremely  broad.  Several  specific  uses  of  the  method  in  this  area 
are: 

,  Establishing  performance  standards  for  use  as  selection 
criteria  in  manning  a  system. 

.  Setting  training  standards  for  BuPers,  P'lee*  Schools 
and  OJT. 

.  Deriving  criteria  for  evaluating  human  engineering  design, 
training  effectiveness  and  personnel  preparedness. 

Developing  sampling  plans  for  proficiency  test  construction 
to  assure  that  the  test  reflects  critical  job  requirements. 

Establishing  qualifications  for  advancement  in  rating  tied 
directly  to  level  of  contribution  to  system  operation. 

.  Stating  alternative  expressions  of  standards  to  meet 

specific  user  needs,  e.g,,  designers,  personnel  research 
personnel,  training  speciadists,  etc. 

.  Developing  occupational  hierarchies  for  personnel  sub¬ 
systems.  (See  Section  VT-C. ) 

B.  System  Design  and  Development 

ihe  basic  applicative:!  of  the  job  standards  derivation  method  in  this 
ov’ca  concern'''  trade-off  analysis  and  testing  compatibility  within 
various  sytem  des'.qn  concepts.  Ihe  major  area  of  attention  would  be 
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personnel  design;  although  consideration  of  equipment  design  and  system 
goals  in  the  trade-off  v-ontext  are  also  amenable  to  analysis. 

Among  the  potential  uses  of  the  method  in  system  design  and 
development  are: 

.  Identification  and  appraisal  of  various  design  and  support 
approaches  for  achieving  stated  system  goals. 

.  Man-machine  function  allocation  trade-off  studies. 

.  Allocation  of  personnel  and  training  resources. 

.  Identification  of  specific  areas  for  emphasis  in  system 

test  and  evaluation  and  for  use-  as  a  basis  for  interpreting 
test  results. 

.  Specification  of  operating  procedures  and  implications 
for  maintenance  activities. 

Also,  the  method  is  equally  applicable  to  existing  systems  when  the 
objective  is  apt  to  be  related  to  determining  and  evaluating  optimal 
approaches  for  upgrading  r.nd  improving  system  effectiveness  under 
relatively  fixed  design  constraints.  Some  specific  areas  of  application 
on  existing  systems  are: 

.  Identification  of  areas  for  emphasis  during  system  evaluating 
and  for  diagnosing  expected  payoff  from  remedial  action. 

.  Testing  the  adequacy  of  existing  personnel  performance 
specifications  in  meeting  system  effectiveness  goais. 

.  Determination  of  the  effects  of  alternative  personnel  per¬ 
formance  levels,  procedures,  sequences,  etc.  on  system 
effectiveness  goals. 

C .  Occupational  Hierarchy  Development 

In  Section  V-A,  it  was  suggested  that  the  job  standards  method  could 
be  extremely  useful  in  developing  occupational  hierarchies  for  personnel 
subsystems.  In  this  section,  a  preliminary  investigation  of  the  potential 
usefulness  of  the  method  for  tliat  purpose  is  reported. 
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1.  Initial  Considerations 


There  are  two  basic  directions  that  could  be  taken  in  using  the  job 
standards  method  to  develop  an  occupational  hierarchy  for  a  system:  (a) 
concentrate  on  the  personnel  needs  of  the  system  to  the  excli^sion  of  any 
existing  occupational  structure;  or  (b)  attend  primarily  to  relating  analyzed 
system  needs  to  an  existing  formal  occupational  hierarchy. 

In  the  first  case,  the  body  of  job  standards  and  behavioral  des¬ 
criptions  could  be  used  to  provide,  ultimately,  a  classification  of  the 
behavioral  demands  on  the  system  to  satisfy  system  goals.  That  use  would 
provide  the  basis  for  a  unique,  current-state  hierarchy  of  Naval  personnel 
position  requirements  if  it  were  integrated  over  a  number  of  similar  Navy 
systems.  If  behavioral  descriptions  from  a  number  of  systems  were 
collated,  the  relations  among  the  behavior  clusters  (positions)  could  lead  to 
an  occupational  hierarchy  based  on  current  system  needs  for  personnel 
performance  rather  than  on  skill  and  knowledge  available. 


The  profit  of  such  a  "system-needs"  oriented  hierarchy  is  a  function 
of  the  manner  in  which  it  would  be  constructed.  As  an  integration  of 
current  and  planned  systems,  it  reflects  the  dynamic  requirements  of 
personnel  activities.  A  formal  occupational  hierarchy  is  principally  a 
reaction  to  technical  needs,  since  it  is  predicated  on  skill  and  knowledge 
availabilities  (for  example,  the  NEC/NOBC).  In  some  cases  both 
hierarchical  types  may  be  coordinated  Into  a  single  occupational  structure 
and,  as  such,  facilitate  personnel  subsystem  allocation,  training  and 
development. 

In  the  second  case,  the  job  standards  method  would  be  used  to 
deveL::p  personnel  position  structure  for  a  given  system  within  the  frame¬ 
work  of  current  perscnnel  classifications.  That  approach  offers  immediate 
profit  and  is  detailed  below.  Regardless  of  w'hich  case  is  selected  finally, 
the  difference  between  cases  is  more  conceptual  than  technical. 

It  was  planned  early  in  the  term  of  the  study,  that  an  occupational 
hierarchy  would  be  developed  for  the  AN/SPS-40  radar  subsystem  as  both 
an  illustration  and  a  test  of  that  use  of  the  job  standards  method.  Due  to 
the  relative  simplicity  of  the  portion  of  the  AN/SPS-40  subsystem  selected 
for  analysis  by  ihe  job  standards  metnod,  particularly  with  respect  to  the 
total  number  of  activities  required  by  the  subsystem,  an  application  of 
the  procedure  described  lelow  for  developing  an  occupational  structure 
did  not  seem  justified.  For  application  and  test  of  the  procedure  to  be 
meaningful,  a  more  comprehensive  body  of  PEF  Units  and  performance 
standards  is  required. 


-  63  - 


2.  Approach 


Our  general  proach  to  the  problem  of  personnel  position 
definition  is  along  traditional  lines,  although  it  differs  in  two  principal 
areas:  (1)  system  data  sources,  and  (2)  rank  ordering  PEF  Units  by  an 
index  of  system  "criticality.”  The  source  of  system  d?  fca  from  which 
personnel  requirements  are  derived  is  not  the  "typical”  system  analysis 
but  rather  the  job  standards.  To  relate  the  personnel  requirements  of  a 
Naval  system  to  personnel  classifications,  the  activities  pnrformed  by 
each  personnel  position  must  be  available.  In  the  job  standards  method 
framework,  those  personnel  activities  are  behaviors  included  within  each 
PEF  Unit,  and  are  obtained  either  from  the  descriptions  associated  with 
each  PEF  Unit  or  from  the  GMDF. 

Data  about  the  types  of  positions  required  by  the  system  are  also 
necessary  to  our  development  of  an  occupational  hierarchy.  That  informa¬ 
tion  is  obtained  from  the  GSSM.  Whether  the  system  being  analyzed  is 
operational  or  still  in  the  design  state,  the  GSSM,  presented  against  a  time¬ 
line,  permits  identification  of  the  minimum  number  cf  positions  necessary. 
That  identification  is  made  without  regard  for  the  general  skills  and 
knowledge  required  by  any  one  PEF  Unit,  and  without  regard  for  special 
or  environmental  constraint  s.  When  those  additional  factors  are  known 
and  relevant,  positional  requirements  may  be  modified.  Factors  of  that 
nature,  when  they  are  identified  as  essential  to  meeting  system  require¬ 
ments,  would  already  be  a  part  of  the  GSSM. 

In  addition  to  the  first  two  kinds  of  information  discussed,  the 
occupational  hierarchy  procedure  requires  a  means  of  ordering  the 
behavioral  descriptions  used,  in  relation  to  occupational  classes.  The 
ordering  is  made  along  the  dimension  of  "criticality.  ”  The  concept  of 
criticality  is  not  new  to  systems  analysis  methodology.  However,  while 
the  concept  receives  little  formal  treatment  in  most  methods,  it  is  an 
integral  part  of  the  procedures  of  the  job  standards  technique. 

It  has  been  mentioned  elsewhere  in  this  report  that  PEF*  Units 
included  in  the  standards  derivation  are  only  those  activities  necessary 
and  sufficient  to  the  goals  cf  the  system.  Therefore,  even  though  PEF 
Units  are  equally  essential  (i.e.,  they  are  necessary  and  sufficient), 
they  may  be  distinguished  by  their  patterns  of  magnitudes  along  sev  ^ral 
dimensions;  as  for  example,  probability  of  required  success,  time,  and 
accuracy.  Positions  of  PEF  Units  on  those  dimensions  permits  ordering 
PEF  Units  to  facilitate  decisions  of  personnel  trade-offs.  The  rank 
order  of  any  one  PEF  Unit  along  such  dim<>r.sions  is  referred  to  here  its 
index  of  criticality  (Crj),  such  that: 

Cri  =  f(P,.  Ti.  Vi.  Ai} 
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where: 

P.  =  the  success  probability  required 
=  the  time  required 

Vj  =  the  recoverability  from  error,  and 

=  the  tolerances,  or  precision.,  required. 

In  other  words,  Cr^  is  principally  a  function  of  the  job  standard 
components  associated  with  each  PEF  Unit,  Thus  far,  in  the  development 
of  the  standards  method,,  the  function  relating  those  factors  has  not  been 
mathematically  expressed  to  the  degree  required  for  use  of  a  criticality 
index.  As  will  be  seen  below,  such  an  index  would  be  used  only  to  order 
PEF  Units,  not  to  quantify  differences  among  them.  The  mathematical 
expression  of  the  criticality  function  need  not,  therefore,  be  highly 
precise. 

3.  Procedure 


An  overview  of  the  procedure  is  shown  schematically  in  Figure  8. 
In  the  figure,  and  in  the  discussion  below,  activities  are  described  in 
procedural  steps. 

Step  1 


Establish  minimum  number  of  personnel  positions.  The  GSSM 
indicates  the  sequence  of  personnel  activities  as  they  are  appropriate  to 
the  system.  To  the  degree  that  activities  are  ordered  sequentially,  they 
constitute  one  position  that  is,  one  person  could  conceivably  perform 
the  activities  (v/ithin  the  limitations  mentioned  in  the  previous  section). 
Each  parallel  display  of  activities  --as  would  follow  the  '  and”  (or  dot) 
symbol,  for  example  --  implies  that  an  additional  position  may  be 
necessary  to  accomplish  tasks  which  can  be  performed  concu  'rently. 

An  additional  position  is  always  imolied  by  the  job  standards  when  the 
time  limitations  for  any  one  sequence  of  activities  is  greater  than  the 
summed  time  for  two  or  more  parallel  sequences.  Both  situations  are 
illustrated  in  Figure  9.  Insofar  as  time  is  concerned,  PEF  Units  1  and  2 
can  be  performed  by  one  person,  while  PEF  Unit  3  constitutes  a  second 
position. 


Step  2 


Obtain  activity  descriptions  for  each  PEF  Unit  personnel  behavior. 
As  has  already  been  mentioned,  the  information  may  be  obtained  from 
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Step  1:  Establish  Minimum  Number  of 
Positions 

V 

Step  2:  Obtain  Activity  Descriptions 


Step  3:  Order  by  Criticality 


V 

Step  4;  Relate  to  Personnel  Qualifications 


Figure  8.  Schematic  of  general  procedure  for  developing 

occupational  hierarchies. 


I 

f 
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Figure  9. 


Hypothetical  partial  GSSM.  ^ 


either  ti  e  GMDF  or  the  Jebcription  associated  with  each  PEF  Unit  standard. 
The  kina  of  activity  description  available  is  contingent,  in  part,  on  tire 
original  purpose  for  which  the  descriptions  were  developed.  That  is,  the 
vocabulary  and  syntax  of  behavioral  descriptions  vary  from  ua-  to  use, 
since  there  is  no  currently  accepted  standard  vocaoulery.  The  vocabulary 
for  job  standards  was  developed  within  the  context  of  various  modeling 
techniques,  with  some  slight  modifications  for  use  with  data  store  informa¬ 
tion.  As  such,  it  is  largely  compatible  with  the  activity  descriptions 
presented  as  part  of  NEC  qualifications 

Step  3 


Order  PEF  Unit  descriptions  by  criticality  for  each  personnel 
position.  The  absence  of  a  firm  method  for  obtaining  a  criticality  index 
has  already  been  discussed;  suffice  it  to  say  here  that  the  index  will 
represent  a  synthesis  of  the  variables  making  up  the  PEF  Units  standards. 
Interpretation  of  the  ordering  is  as  follows;  while  all  PEF  Units  are 
necessary  to  successful  satisfaction  of  system  requirements,  the  more  a 
PEF  Unit  is  ranked  as  critical  *’hr  more  important  it  is  that  that  PEF  Unit 
be  accomplished  correctly  in  -trict  accord  with  the  established  job 
standards:  because  of  (1)  its  irretrievable  nature,  (2.)  a  low  initial 
probability  of  success  --  the  lower  the  probability  of  success,  the  greater 
the  impact  on  system  performance  of  any  change  in  probabilities,  (3)  the 


State  descriptions  have  been  omitted  for  the  sake  of  brevity. 
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impact  on  system  performance  of  a  change  in  its  probability  of  accomplish¬ 
ment,  or  (4)  the  dependence  of  several  other  PEF  Units  on  successful 
completion  of  the  PEF  Unit  in  question.  All  such  factors  are  reflected  in 
a  "criticality"  ranking  of  PEF  Units.  Specific  utility  of  this  step  will 
become  clear  as  the  final  step  is  discussed. 

Step  4 


Compare  PEF  Unit  activities  and  rating  qualification  statements 
(in  NAVPERS  18068A).  Those  qualifications  for  advancement  appear,  in 
part,  as  a  series  of  factors  within  areas,  such  as  (1)  theoretical  and 
practical  equipment  knowledge,  (2)  safety,  operating  and  administrative 
procedures,  and  (3)  maintenance.  Within  each  area,  behavior  statements 
which  describe  a  particular  qualification  are  also  presented,  with  the 
advancement  level  to  which  the  qualifying  behavior  is  relevant.  While  they 
are  usually  presented  by  factor,  they  can  easily  be  reorganized  by  rating 
level  (within  specialist  codes)  to  facilitate  use  in  personnel  assignment. 
Regardless  of  the  format  in  which  the  qualification  statements  appear, 

PEF  Unit  activities  and  the  qualification  statements  are  compared  to 
identify  the  qualification  statement  most  like  a  given  PEF  Unit  activity  in 
all  accountable  respects.  When  a  PEF  Unit  has  been  identified  as  most 
related  to  a  given  qualification  statement,  the  associated  rating  lovel  is 
noted.  The  list  of  ratings  which  results  constitutes  the  personnel  require¬ 
ments  for  the  system  in  question. 

When  ‘he  steps  above  have  been  completed  a  summary  table,  such 
as  Table  2,  may  be  constructed.  The  table  displays  direct  information 
regarding  the  levels  and  kinds  of  specialist  personnel  required  to  man  the 
system  --by  PEF  Unit  as  well  as  by  position.  The  table  also  presents  the 
rating  distribution  of  skill  levels  for  each  position.  In  the  example  given, 
all  activities  for  Position  A  may  be  performed  by  an  RD2.  Position  B, 
however,  has  activities  that  are  within  the  skill  levels  of  several  RD 
ratings.  If  an  RD3  were  given  the  position,  PEF  Units  4  and  6  would  be 
beyond  his  skill  level  --he  would  n^ed  at  least  OJT  to  bring  him  up  to  the 
required  level  of  performance.  At  the  other  extreme,  while  a  RDl  has 
all  the  skills  necessary  to  fulfill  the  performance  requirements,  he  is, 
to  a  degree,  overskilled  fer  the  demands  of  the  particular  position.  In 
this  example,  the  most  critical  PEF  Unit  for  Position  B  is,  by  definition, 
PEF  Unit  4.  According  to  the  table,  that  activity  can  be  performed  by  a 
RD2.  The  manning  decision  might  be  made,  therefore,  to  specify  an 
RD2  as  minimum  for  the  position,  recognizing  that  he  is  ov'ertrained  for 
at  least  one  activity  and  will  require  OJT  for  another  activity.  Without 
additional  training,  he  can  complete  66.6%  of  the  activities  required, 
including  the  two  most  critical  ones. 
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Table  2 

Hypothetical  Relation  of  System  Personnel  Requi;*ements  and  Ratings 


Rating 
(for 
radar - 
man) 

Position  A 

_ 

Position  B 

PEF  Unit 

1 

PEF  Unit 

2 

PEF  Unit 

3 

PEF  Unit 

4 

PEF  Unit 

5 

PEF  Unit 

6 

RDC 

RDl 

RD2 

RD3 

X 

X 

X 

X 

X 

X 

4.  Discussion 


The  principal  use  to  which  the  occupational  hierarchy  development 
procedure  may  be  applied  is  in  effective  assessment  of  the  personnel 
requirements  of  a  system  as  outlined  in  the  preceding  section.  With  the 
exceptions  noted  earlier,  use  of  system  personnel  activities  for  that  purpose 
is  not  unique  to  standards  methodology. 

The  highly  quantitative,  system-activity -oriented  data  yielded  b/ 
the  procedure  are  unique.  Immediate  use  uf  the  data  may  be  made  in 
several  potentially  profitable  ways,  the  foremost  being  in  an  attempt  to 
assecs  the  overall  contribution  of  a  particular  position  to  the  operational 
success  of  the  system.  Once  personnel  requirements  have  been  determined, 
it  would  be  a  relatively  simple  matter  to  compute,  for  example: 

.  Proportion  of  activities  performed  (operator  loading)  by 
any  one  position,  rating,  group  or  individual  PEF  Unit  -- 
in  any  combination, 

.  Manning  costs,  and 

.  Contribution  of  any  position,  PEF  Unit  or  rating  to 
overall  system  effectiveness. 

Because  the  general  occupation  allocation  procedure  has  not  yet 
been  tested  fully  the  examples  above  are  mentioned  only  briefly  to 
illustrate  the  contribution  to  system  planning  that  the  procedure  could  make. 
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VII.  ADDITIONAL  RESEARCH  RECOMMENDED 


As  was  noted  in  Section  I-B,  the  present  study  was  devoted  to  develop¬ 
ing  a  basic  capability  for  deriving  system-relatt  d  job  standards  and  for 
testing  the  effects  of  performance  levels  that  devk.te  from  established  job 
standards  on  system  effectiveness.  Described  below  are  several  logically 
relatv^d  task  areas  that  are  necessary  to  extend  and  refine  that  basic 
capability  through  additional  methodological  research,  empirical  testing 
and  specific  treatment  of  implementation  and  application  proolems.. 

A.  Methodological  Refinement 

For  the  job  standards  method  to  realize  its  full  potential,  it  is  necessary 
that  methodological  research  be  continued  to  extend  the  basic  capability  of 
the  method  described  in  this  report.  The  applicability  of  other  mathematical 
techniques  and  procedures  should  be  explored  as  well  as  further  and  more 
comprehensive  testing  of  the  method  using  actual  system  data. 

1.  Mathematical  Development 

Additional  research  is  required  in  the  area  of  requirement  alloca¬ 
tion  techniques.  It  is  evident  that  increased  reliability  and  comprehensiveness 
in  the  basis  for  arriving  at  an  index  of  accomplishment  for  the  personnel/ 
equipment  functional  units  is  desirable.  Preferably,  the  indices  would  be 
based  on  objectively-defined  characteristics  of  any  system  in  its  dynamic 
operation,  rather  than  on  discrete  independent  elements  of  system 
functioning.  Th^  indices  should  be  able  to  serve  at  least  three  functions: 

.  As  weighted  values  for  deriving  the  probability 
components  of  personnel  performance  standards 
via  the  MSSM. 

.  As  a  basis  for  further  development  and  verification 
of  the  techniques  by  utilizing  feedback  information 
from  observations  implied  in  the  derivation  of  the 
indices , 

,  As  a  potential  basis  for  evaluating  (and  incorporating 
where  appropriate)  human  capability  data  from  other 
sources . 

The  Index  of  Task  Accomplishment  (IOTA)  eventually  should  be  a. 
multi-dimensional  variable  whose  primary  dimensions  involve  relatively 
stable  characteristics  of  a  given  system,  including  man,  for  any  duration 
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and  accuracy  of  operation.  That  is,  for  given  time  and  accuracy  constraints 
human  and  equipment  characteristics  are  expected  to  vary  extremely  slowly 
compared  to  overall  system  performance.  However,  different  time  and 
accuracy  con.traints  may  result  in  significant  changes  in  index  values, 
particularly  as  those  constraints  become  psychologically  or  physiologically 
stressful.  Although  this  area  has  been  investigated  and  several  tentative 
hypotheses  have  been  generated  (See  Section  IV-B),  additional  effort  i.s 
recommended  to  refine  and  test  those  hypotheses. 

The  present  method  rest’s  on  the  application  of  conventional  con¬ 
ditional  probability  theory  to  relate  defined  states  of  the  system  through¬ 
out  its  operation.  There  are  other  probability  techniques  for  relating 
system  states  depending  upon  how  they  are  to  be  used  and  the  kinds  of 
operations  to  which  they  relate.  Those  techniques  should  be  evaluated 
and  if  they  prove  useful,  th  ^  should  be  incorporated  into  the  overall 
approach. 

2.  Integration  of  Maintenance  Considerations 

In  the  present  study,  it  ’./as  determined  that  overall  maintenance 
time  requirements  can  be  established  readily  from  the  system  availability 
requirements  if  system  failure  rate  is  knovm.  Also,  it  was  determined 
that  time  requirements  could  be  established  for  various  categories  of 
maintenance,  defined  in  accordance  with  demands  on  maintenance  per¬ 
sonnel  capabilities.  In  addition,  it  was  found  that  maintenance  actions 
can  be  modeled  using  graphic  and  mathematical  state  sequence  modeling 
techniques  which  constitute  the  key  elements  of  the  basic  standards 
meth'^d. 


Additional  effort  is  recommended  to  integrate  maintenance  t.on- 
siderations  completely  in  the  standards  method,  and  should  include: 

,  Performance  of  a  detailed  analysis  of  the  general 
maintenance  model, 

,  Determination  of  the  most  appropriate  classification 
system  for  establishing  detailed  maintenance  time 
standards, 

.  Determination  of  an  appropriate  index  of  task 

accomplishment  for  allocating  standards  to  categories 
of  maintenance.  (It  may  be  determined  that  one  index 
of  task  accomplishment  may  be  appropriate  for  allocating 
both  operator  and  maintenance  PEF  Unit  standards.) 
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3. 


Alternative  Job  Standard,  Expressions 

As  presently  conceived,  a  job  standard  ultimately  will  be  com¬ 
posed  of  a  personnel/equipment  functional  unit  {PEF  Unit),  an  accuracy/ 
time  requirement  and  a  required  probability  of  successful  performance. 
Alternative  expressions  of  a  performance  standard  may  be  required  to 
fulfill  the  specific  needs  of  several  pc  ntial  user  groups.  For  example, 
a  training  specialist  may  be  concerned  mainly  with  a  stated  criterion 
performance  level  Ox-  interval  of  performance  which  his  training  program 
must  meet.  On  the  other  hand,  a  systems  design  specialist,  i  oncerned 
with  trade-off  analyses,  would  be  likely  to  require  a  probability  dis¬ 
tribution  or  expectancy  data  to  be  associated  with  a  performance  standard. 

The  capability  to  develop  alternative  job  standard  expressions  is 
dependent  upon  progress  in  expanding  the  definition  of  a  job  standard  to 
include  various  weighted  combinations  of  objective  measures,  maximum/ 
minimum  performance  statements,  probability  distributions  as  functions 
of  accuracy  or  time  permitted  and  other  distributed  functions. 

4.  Empirical  Test  of  the  Method 


Data  obtained  on  the  AN/SPS-40  radar  subsystem  were  employed, 
in  the  present  study,  to  provide  for  on-going  test  and  refinement  of 
various  elements  of  the  method  being  developed.  Use  of  actual  data  as 
an  integral  part  of  the  development  process  was  found  to  be  a  great  value 
and  it  is  recommended  that  that  approach  be  continued  in  future  work. 

Two  types  of  tests  could  be  employed:  ver*ical  and  horizontal. 

The  vertical  test  consists  of  examining  the  method  within  the  context  of 
a  single  system  such  as  AAW  or  ASW  and  its  subsystems.  (The  AN/SPS-40 
constitutes  a  subsystem  of  an  AAW  system.  )  The  horizontal  test,  on  the 
other  hand,  involves  examination  of  the  method  across  systems  at  the 
system  oj  subsystem  level.  (For  example,  considering  subsystems  other 
than  the  AN/St^S  -40  such  as  a  navigation  or  sonar  s  ibsystem. )  At  presen*, 
it  appears  that  vertical  applicability  of  the  method  should  be  given  greater 
emphasis  than  horizontal  applicability.  Therefore,  it  is  recommended 
that  test  coverage  be  broadened  within  the  AAW  system  context  by 
considering  other  subsystems  in  conjunction  with  the  AN/SPS-40. 

B.  Method  Implementation 


The  problem  of  implementing  a  te  'hnique  as  comprehensive  as  the 
job  standards  method  wdth  its  broad  application  is  an  important  cne.  A 
carefully  developed  a  id  thoroughly  tested  method  would  have  little  utility 
if  implementing  it  is  time  consuming  and  laborious  and  its  integration 
with  existing  programs  is  poorly  effected.  Consequently,  this  area  deals 
with  additional  research  designed  to  circumvent  those  potential  difficulties. 
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1,  Procedural  Format 


One  of  the  products  of  the  present  study  is  a  procedural  format 
that  ultimately  will  guide  the  user  on  a  step-by-step  basis  in  applying 
the  method.  As  the  method  is  refined,  modified  and  improved,  the 
procedural  format  must  also  be  revised  to  reflect  any  changes  and 
refinements.  In  addition,  the  procedural  format  should,  eventually  be 
tested  using  a  sample  of  Naval  system's  analysts  to  assure  t’^at  it  fulfills 
its  purpose  adequately.  Ultimately,  a  user’s  guide  for  the  jot  standards 
method  should  be  produced  that  will  instruct  the  user  in  all  aspects  of 
integrating  the  method  in  system  design,  development  and  test  efforts, 
specify  data  requirements,  computer  applications,  procedural  steps, 
output  interpretation  and  reporting  techniques, 

2.  Integration  with  the  Navy’s  RDT&;E  Process 

If  the  potential  of  the  job  standards  method  as  a  system  design 
and  development  tool  is  to  be  reaiiz,ed,  definite  steps  must  be  taken  co 
integrate  it  into  the  Navy's  RDT&E  process.  One  of  the  first  steps  in 
achieving  that  integration  is  to  specify  the  input  data  requirements 
necessary  for  the  method  to  be  used  to  establish  performance  standards 
and  to  evaluate  design  trade-offs.  Effort  is  recommended  to  develop  a 
complete  data  requirements  package  that  will  detail  the  types  of  data 
required,  levels  of  specificity,  quantification  form  and  when  and  how  they 
should  be  collected  and  submitted  during  Naval  system  design  and  develop¬ 
ment.  By  specifying  the  anticipated  data  requirements  of  the  job  standard 
method  as  early  as  possible  in  its  development,  effective  introduction  of 
the  method  into  the  Navy's  RDT&E  process  can  be  achieved  with  minimum 
time  delay. 

3.  Use  of  Computer  Techniques 


Application  of  the  job  standards  method,  especially  in  design  trade¬ 
off  analyses,  v/ill  involve  a  great  deal  of  calculation.  The  greater  the 
number  of  approaches,  contingencies,  qualifications  and  restrictions  that 
need  to  be  explored,  the  more  complex  the  mathematical  models  will 
become.  To  obtain  results  rapidly  and  efficiently,  computerization  of  the 
mathematical  operations  should  be  considered. 

Witn  the  results  of  further  development  of  the  mathematical  aspect 
of  the  method  in  hand,  effort  should  be  initiated  to  program  those  portions 
of  the  job  standards  method  which  involve  substantial  computation;  i.e.  , 
those  which  have  been  found  most  laborious  and  time  consuming  using  manual 
techniques.  It  is  likely  that  existing  computer  programs  will  be  applicable 
for  some  of  the  operations  and  any  additional  programs  that  might  be 
required  should  be  relatively  simple  to  design. 
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Vni.  GLOSSARY 


A  Accuracy 

Crj  A  criticality  measure  of  the  ith  PEF  Unit 

f{  )  An  unspecified  function  of  the  variable(s)  contained 

in  the  parentheses 

g(  )  An  unspecified  function  of  the  variable(s)  contained 

in  the  parentheses 

GMDF  Graphic  Model  Data  Form 

GSSM  Graphic  State  Sequence  Model 

i  (1)  A  symbolic  index  used  to  represent  an  integer  or 

alphabetical  character;  it  refers  to  one  of  a  set 
of  things, 

(^)  The  variable  IOTA 

That  value  of  i  (or  IOTA)  when  there  is  no  time  constraint 
and  the  tolerance  requirement  --  if  one  exists  --  is  not 
varied. 

The  input  state  of  the  ath  PEF  Unit 
IOTA  Index  of  Task  Accomplishment 

j  A  symbolic  index  used  to  represent  an  integer  or 

alphabetical  character;  it  refers  to  one  of  a  set  of  things 

J  Difficulty  of  a  job;  a  discrete  variable 

k  The  constant  for  converting  an  IOTA  value  into  the 

probability  of  accomplishing  the  associated  PEF  Unit 

K  A  proportionality  constant  associated  with  alternative 

procedures;  the  sum  of  the  K  values  following  a  single 
CROSSED  DOT  symbol  in  the  GSSM  equals  1.0, 

L  A  required  level  of  measurement  tolerance  (inversely 

related  to  accuracy) 


- 


m  (1)  The  uppermost  of  a  number  of  values  under  con¬ 

sideration;  an  integer  or  alphabetical  character 
referring  to  the  number  of  things  in  the  set  under 
consideration 

(2)  The  number  of  repetitions  of  a  set  of  PEF  Units 

(3)  One  of  a  set  of  continuously  variable  measures 
(used  in  Section  IV-B-5) 

m  The  mean  of  a  distribution  of  m  measures 

m'  One  of  a  set  of  continuously  variable  measures  having 

m  =  0 

M  Man’s  capability;  a  discrete  variable 

MSSM  Mathematical  State  Sequence  Model 

MTBF  Mean  Time  Between  Failures 

MTTR  Mean  Time  to  Restore 

n  The  uppermost  of  a  number  of  values  under  >mnside ration; 

an  integer  or  alphabetical  character  referring  to  the 
number  of  things  in  the  set  under  con  side  ration 

Ojj  The  output  state  of  the  a  th  PEF  Unit 

P  Probability  --  often  used  instead  of  P(  ),  where  the 

parenthetical  portion  is  understood 

P^  Probability  cf  accomplishing  the  i  ch  PEF  Unit 

P(  )  Probability  of  the  occurrence  (or  of  accomplishing) 

whatever  is  contained  within  the  parentheses 

P(in)  Probability  of  the  input  state 

P(out)  Probability  of  the  output  state  of  the  system  or  a  portion 

or  of  the  system.  Usually,  it  is  understood  that  its  value 

P^5  is  given  as  a  system  effectiveness  requirement 

PEF  Unit  Personnel/Equipment  Functional  Unit 

PjE^  The  per.»cnnel  and  equipment  included  in  the  i  th  PEF  Unit 


Rjl^  Time  to  carry  out  maintenance  on  the  ith  hardware  item 

An  estimate  of  system  output  based  on  actual  or  proposed 
capabilities  for  performing  component  PEF  Units 

Sj.  A  system  effectiveness  requirement 

SER  System  Effectiveness  Requirement 

t  Time  (variable) 

T^  Time  required  to  accomplish  the  ith  PEF  Unit 

Tmax  ^  maximum  value  of  time,  t 

Tmin  ^  minimum  value  of  time,  t 

Tj  The  total  time  allowable  for  the  system  to  reach  its 

output  state;  a  system  effectiveness  requirement 

Vi  Recoverability  from  error  of  the  ith  PEF  Unit 

X  (1)  Used  in  Appendix  B  to  represent  a  PEF  Unit 

(2)  A  symbolic  index  used  to  represent  an  integer  or 
alphabetical  character;  it  refers  to  a  set  of  things 

X  A  symbol  used  in  examples  to  represent  an  unspecified 

piece  of  equipment  or  an  unspecified  human,  depending 
upon  the  ccntext 

y  Used  in  Appendix  B  to  represent  an  input  state 

z  Used  in  Appendix  B  to  represent  an  output  state 


a  A  symbolic  index  used  to  represent  an  integer  or 

alphabetical  character;  it  generally  *'efers  to  one  of 
a  set  of  PEF  Units 

^  A  symbol  representing  a  difference  between  two  values; 

e.  g.  ,  At  may  indicate  an  interval  of  time  between  t|^ 
and  t^ 
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c  A  small  positive  increment  of  a  numerical  value 

Xj  Failure  rate  of  the  i  th  hardware  item 

TT  3.14159 

n  Product  symbol,  indicating  that  all  values  following  the 

symbol  are  to  be  multiplied  together 

(T  The  standard  deviation  of  a  Gaussian  (normal)  distribution 

S  Summation  symbol,  indicating  that  all  values  following 

the  symbol  are  to  be  added  together 

T  The  tii/ie  variable  upon  which  a  mathematical  operation 

is  performed 

T  The  mean  of  a  set  of  T  values 

<  Less  than. . . 

<  Less  than  or  equal  to  . . , 

>  More  than  . . . 

>  More  than  or  equal  to  .  . . 

oo  Infinity 

Note:  GSSM  symbols  are  defined  in  Figure  Z,  page  18. 
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PROCEDURE  FOR  DEVELOPING  PERSONNEL  PERFORMANCE  STANDARDS 


I.  INTRODUCTION 


A.  General  Remarks 


The  purpose  of  this  Appendix  is  to  present  preliminary  procedural 
steps  and  rules  for  implementing  a  method  for  establishing  personnel 
performance  standards  to  meet  system  effectiveness  requirements. 
Since  the  method  is  still  in  the  developmental  stage,  the  procedures 
are  rough  and  incomplete;  much  more  work  needs  to  be  done  before 
they  can  be  refined.  However,  they  reflect  the  progress  that  has  been 
made  to  date. 

The  organization  of  subsequent  sections  of  this  Appendix  follows 
the  general  outline  of  approach  shown  in  Figure  I-l.  Use  of  the  pro¬ 
cedures  included  here  rests  primarily  upon  three  basic  assumptions: 

(1)  the  user  has  experience  in  systems  analysis,  in  general;  (2)  the 
user  is  very  familiar  with  the  specific  system  being  analyzed,  to  the 
extent  that  he  can  fully  utilize  the  available  data  describing  the  system, 
and  (3)  the  procedures  are  to  be  applied  to  the  analysis  and  evaluation 
of  existing  systems,  in  contrast  to  aiding  design.  ^  The  requirement 
for  familiarity  with  the  specific  system  is  important  because  the 
analyst  must  be  able  to  perceive  the  system's  capabilities  objectively 
without  being  constrained  by  the  performance  expected  of  it.  1  hat  is, 
he  will  be  called  upon  to  recognize  if  the  system  might  perform  in  ways 
other  than  those  (1)  recorded  in  documents  such  as  operational  proce¬ 
dures,  or  (2)  carried  out  in  actual  practice. 

Our  technique  depends  primarily  upon  a  knowledge  of  general 
procedural  rules  which  may  apply  at  any  time  throughout  the  con¬ 
struction  process.  Examples  of  application  are  presented.  Common 
situations,  rather  than  specific  systems  or  subsystems,  are  employed 
as  sample  personnel/equipment  functional  units  (PEF  Units). 

B.  Definitions 


System  -  A  system  is  an  operational  set  of  components 
delineated  by  its  input  state,  required  output  state,  and 
constraints.  A  system  includes  all  features  of  the  set 
which  are  effected  by  the  transition  Irom  input  state  to 


Procedures  for  utilizing  the  method  in  designing  new  systems  cf.n  also 
be  developed,  and  would  constitute  a  logical  extension  of  the  work 
described  in  the  body  of  the  report. 
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Figure  I-l.  Outline  of  approach. 


output  state  within  the  constraints  imposed  upon  it.  A 
system  can  be  represented  graphically  as  a  single  (com¬ 
plex)  personnel/equipment  functional  unit  (PEF  Unit). 

PEF  Unit  -  A  personnel/ equipment  functional  unit  is  a 
general  term  indicating  man-machine  interaction  at  any 
level  of  specificity,  depending  upon  the  context  of  dis¬ 
cussion.  As  opposed  to  a  state,  a  PEF  Unit  represents 
one  or  more  operations,  activities  or  processes. 

State  -  A  state  of  a  system  i.s  identified  by  a  complete 
description  of  the  relevant  and  measurable  characteristics 
of  the  system  at  an  instant  in  time.  (The  description  of 
the  state  is  always  a  declarative  sentence  including  the 
word  "is"  or  "are",  e.  g.  ,  height  and  weight  of  pilot  X  are 
recorded  on  form  P.  ) 

•  The  input  state  of  the  system  is  specified  by  initial 
system  conditions  which  can  undergo  change  during 
system  performance;  a  change  may  occur  at  any  time 
in  the  system's  operation  until  the  output  state  is 
reached. 

•  Relevant  features  of  Intermediate  states  (between 
two  or  more  PEF  Units)  are  all  the  instantaneous 
conditions  of  the  system  which  have  undergone 
change  as  a  result  of  the  immediately  preceding 
operation  (1.  e.  ,  prior  PEF  Unit).  While  only 
changed  features  are  described  for  intermediate 
system  states,  all  prior  conditions  are  assumed. 

That  is,  the  description  of  a  state  implicitly  in¬ 
cludes  all  unchanged  portions  of  the  system  input 
state  and  the  last-described  condition  of  all  features 
which  have  been  altered  from  the  system  input  state. 

•  The  output  state  is  specified  by  the  overall  system 
requirements.  It  includes  objective  statements 
indicating  the  accomplishment  of  the  system’s 
objectives. 

F unction  -  The  broadest  description  of  system  operations 
is  made  in  terms  of  functions.  A  function  is  a  PEF  Unit 
which  is  immediately  related  to  the  system  mission  such 
that  a  more  generab  level  of  system  operation  cannot  be 
specified.  The  system  function  is  developed  apart  from 
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and  without  regard  to  the  specific  hardware  of  the  system 
being  described.  As  a  rule,  systems  with  similar  mis¬ 
sions  will  have  similar  functions. 

•  Element  -  An  element  is  alternatively  called  an  elemental 
PEF  Unit.  It  represents  the  most  specific  level  of  system 
description  to  which  these  procedures  are  designed  to  be 
applied.  The  complete  definition  of  an  element  was  devel¬ 
oped  in  accordance  with  the  rules  for  constructing  a 
Graphic  State  Sequence  Model  (GSSM).  Those  rules  appear 
in  Section  HI.  (The  remainder  of  Section  I  is  a  detailed 
definition  and  explanation  of  an  element.  The  reader  may 
skip  to  Section  11  and  study  Sections  II  and  III  before  com  - 
pleting  this  section.  ) 

1.  The  PEF  Unit  involves  an  interaction  between  a  human 
and  either  (a)  an  inanimate  feature,  or  (b)  another  human. 
Reiteratively,  the  action  does  not  lie  solely  within  a  sin¬ 
gle  human  (e.  g.  ,  thought  or  pure  muscular  response, 
such  as  lifting  the  arm)  nor  dees  it  lie  solely  within  the 
system  (e.  g.  ,  current  supplied  to  a  circuit). 

2.  One  or  more  of  the  following  conditions  must  apply: 

(a)  The  input  state  of  the  PEF  Unit  is  the  system  input 
state. 

(b)  The  output  state^  of  the  immediatelv  preceding  PEF 
Unit  has  more  than  one  defined  part^  or  alternative, 
not  all  of  which  form  all  or  part  of  the  input  state  of 
the  PEF  Unit  under  consideration. 

(c)  The  input  state  includes  a  feedback  component  which 
is  not  of  a  "re-do"  type,  i.  e.  ,  the  feedback  does  not 
imply  that  the  PEF  Unit  is  to  be  performed  either 


Throughout  the  definition,  "output  state"  refers  only  to  that  required 
by  the  system. 

Whenever  the  word  "part"  is  used  in  connection  with  the  input  or  out;  ■ 
state,  we  assume  the  diagrammer's  vantage  point.  "Part"  refers 
either  to  (a)  a  portion  of  the  total  required  state,  or  (b)  one,  two  or 
more  various  required  states. 
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because  the  action  has  not  yet  been  completed  or 
because  it  was  not  done  properly. 

(d)  The  input  state  comprises  more  than  one  component, 
each  of  which  represents  at  least  part  of  the  output 
state  of  another  PEF  Unit. 

(e)  The  input  state  is  identical  to  the  output  state  of  an 
immediately  preceding  PEF  Unit  only  if  one  of  the 
following  applies: 

(1)  The  unit  under  consideration  fulfills  require¬ 
ment  4{b). 

(2)  The.  immediately  preceding  PEF  Unit  fulfills 
requirement  4(b), 

3.  Somewhat  overlapping  Condition  No.  2,  one  of  the  following 
must  apply: 

(a)  The  output  state  of  the  PEF  Unit  is  the  system  output 
state. 

(b)  The  output  state  forms  part  of  the  input  state  to  a 
subsequent  element. 

(c)  A  part  of  the  output  state  forms  aT.  or  part  of  the 
input  state  to  a  subsequent  element. 

(d)  The  total  output  state  forms  the  total  input  state  to 
a  subsequent  element  :‘nly  if  one  of  the  following 
applies: 

(1)  The  unit  vender  consideration  fulfills  require¬ 
ment  4(b). 

(2)  The  subsequent  element  fulfills  requirement 
4(b). 

4.  One  of  the  following  conditions  must  apply: 

(a)  Any  further  fractionation  of  the  PEF  Unit  under  con¬ 
sideration  will  result  in  a  failure  to  meet  one  or  more 
of  the  requirements  above. 
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(b)  Further  frictionation  of  the  PEF  Unit  under  con¬ 
sideration  fails  to  meet  requirements  2{b),  2(c) 
and  2(d)i  and/or  3(b)  and  3(c),  but  the  same  PEF 
Unit  appears  somewhere  else  in  the  system 
description  .ere  it  satisfies  requirements  1, 

2(a)  through  <:(d),  3(a)  through  3(d),  and  4(a), 

'^o  illustrate  application  of  those  criteria,  a  hypothetical  clement 
GSSM  is  shown  in  skeletal  form  in  Figure  1-2. 


Figure  1-2.  Hypotheti,  :ai  skeletal  GSSM  illustrating  the 
criteria  for  depictii'g  a  PEF  Unit  at  the 
elemental  level. 


It  is  assumed  that  the  diagram  represents  a  system,  or  a  portion 
of  a  system,  and  that  all  PEF  Units,  A  through  G,  involve  a  man's 
inte  ‘action  with  'ither  a  piece  of  equipmtnt  or  another  man  (thereby 
satisfying  requirement  No.  1). 

The  PEF  Units  are  connected  by  lines  which  represent  the  system 
slates  at  the  respective  points  in  the  system  operation.  It  may  be 
noted  that  there  are  two  possible  output  states  after  PEF  Uiiit  D, 
only  one  of  which  can  occur  during  a  single  operation  of  the  system; 
one  aJtern^.tive  is  the  input  state  for  E,  the  other  is  the  simultaneous 
input  state  for  B  and  F. 

The  numbers  and  letters  which  appear  on  the  connecting  lines  in 
Figure  1-2  reference  paragraphs  in  the  criterion  definition  for  an 
element  PEF  Unit.  As  an  example,  therefore,  the  input  state  for 
A  satisfies  requirement  2(a),  whil?  the  output  state  for  A  reflects 
paragraph  3(d)(2).  It  will  be  noted  that  if  2(e/,  3(d)  and  4(b)  were 
omitter,  the  string  of  units,  A-B-C,  would  not  satisfy  No.  2  or  3, 
thereby  demonstrating  the  effect  of  4(a).  However,  B  occurs 
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elsewhere  in  the  system  so  that  it  fulfills  the  requirements  of  4(b). 

We  assume  that  we  can  demonstrate  that  none  of  the  other  PEF  Units 
except  B  can  be  fractionated  further  without  violating  one  or  more  of 
the  requirements. 

The  importance  of  delineating  meaningful  elements  of  system  per¬ 
formance  rests  on  the  fact  that  human  activity  can  be  reduced  theo¬ 
retically  to  an  underlying  set  of  neuromuscular  and  molecular*  changes. 
However,  one  might  spend  much  time  trying  to  define  a  system  at 
that  level  without  ever  arriving  at  a  means  for  establishing  standards. 
It  is  necessary,  therefore,  to  state  system  behavior  only  to  that 
describable  and  measurable  level  necessary  and  amenable  to  an 
effective  methodology.  In  our  approach,  the  element'  PEF  Unit  repre¬ 
sents  that  minimum  level;  it  is  definable  according  to  non-behavioral, 
.elatively  objective  criteria  relating  to  construction  by  logical  dia¬ 
gramming.  Any  further  attempt  at  moving  to  a  lower  level  woulo 
have  no  effect  on  the  basic  structure  of  the  descriptive  mathematical 
model  we  wish  to  derive  from  the  GSSM  at  the  element  level.  It  is 
the  mathemaHcal  model,  not  the  GSSM,  which  provides  the  basis 
for  determining  probabilities  (for  standards)  associated  with  system 
performance  requirements,  so  procedural  efficiency  is  unnecessarily 
abused  by  erecting  a  more  complex  GSSM  than  is  required  by  the 
mathematical  model  we  wish  to  derive.  How'ever,  since  the  mathe¬ 
matical  model  will  be  derived  directly  from  the  graphic  model,  the 
latter  must  be  complete  and  accurate  for  the  mathematical  state-  lent 
to  represent  the  system  faithfully. 
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II.  ORGANIZING  DATA  ON  GRAPHIC  MODEL  DATA  FORMS 


Pj^eparation  and  Purpose 

Construction  of  the  Graphic  Model  Data  Form  (GMDF)  is  an  organiza¬ 
tional  process.  It  depends  upon  thorough  familiarity  with  the  system.  In 
order  to  proceed#  the  analyst  must  nave  accejs  to  all  available  system 
descriptive  material,  including  its  operational  requirements.  {See  Step  1 
under  B  below.  ) 

The  purpose  of  the  GMDF  is  to  organize  whatever  information  is  already 
generated  regarding  the  human  contribution  to  system  performance  in  a 
format  that  will  facilitate  subsequent  modelling  operations.  Generally,  the 
information  necessary  for  constructing  the  GMDF  is  likely  to  be  available 
in  the  form  of  operational  procedures  fcr  a  system  which  already  exists. 

B.  Procedural  Steps  at  the  Function  I.evel 

Step  J 

Prior  to  using  the  Data  Forms,  list  the  following  information: 

a.  Overall  system  effectiveness  requirements 

•  The  precise  output  (final)  state  of  the  system 

•  Tolerances,  or  accuracy  requirements,  associated 
with  each  feature  of  the  output  state 

•  Time  limitations  from  onset  of  system  functioning 
(if  applicable) 

•  Probability  required  for  the  output  steito  components 
and  their  toler'^nces  to  exist  when  the  system  has 
completed  its  functioning 

b.  Input  state  conditions,  i.  e.,  complete  descriptions  of  the 
relevant  system  just  prior  to  its  initiation.  ^ 

c.  Complete  characteristics  of  the  system  at  any  intermediate 
stage  of  its  operation  for  which  effectiveness  requirements 
have  been  established. 


See  definition  of  input  state. 
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•  Describe  altered  conditions  (from  that  in  Step  l.b. 
and  from  prior  intermediate  conditions) 

•  List  measurable  characteristics  of  the  system  with 
tolerance  requirements 

•  Time  limitation,  if  applicable 

•  Probability  required  to  achieve  the  specified  system 
state 

d.  System  constraints,  either  specified  or  assumed. 

Step  Z 

Figure  11- 1  illustrates  the  recommended  headings  for  the  GMDF 
columns.  The  meaning  of  those  headings  will  be  clarified  in  the 
subsequent  discussion. 


Graphic  Model  Data  Form 


Function 


PEF 

Unit 

Number 

Prior 

Necessary 

Step(s) 

Alter¬ 

nate 

Step(s) 

Activity  Description 

Output 

State 

Imposed 

Require¬ 

ment 

Figure  II-l.  Recommended  headings  for  the  GMDF. 


On  the  first  GMDF,  cross  out  'Function”  and  write  "System”  at  the  top. 
This  data  form  will  contain  the  initial  breakdown  of  the  system  into  Functions. 
Divide  the  system  into  as  many  Functions  as  possible,  in  accordance  with  the 
definition  in  Section  I,  B,  (Some  or  all  of  the  system  conditions  between  func¬ 
tions  may  already  have  been  delineated  in  Step  l.c. ) 
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a.  Assign  a  capital  letter  to  each  function,  starting  with  "A'' 

and  place  those  letters  under  ti}e  heading  "PEF  Unit  Number.  ' 

b.  Under  "Prior  Necessary  Step(s)"  list  the  function(s)  requirt^d 
in  order  that  the  needed  input  state  exists.  For  example,  if 
PEF  Units  B  and  C  are  parallel  functions  coming  after  A,  the;- 
the  prior  necessary  step  for  both  B  and  C  will  be  "A",  bec  ause 
A  provides  the  require^a  input  state  for  C  as  well  as  for  ;i. 

(See  relation  No.  3  in  Figure  II1-2  on  page  B-16.  )  Simil  irl^, 

if  A  and  B  together  produce  the  input  state  for  C,  then  A  and  B 
are  both  listed  as  the  "Prior  Necessary  Step(s)"  for  C.  (See 
relation  No.  2  in  Figure  IiI-2.  ) 

c.  List  whether  alternate  functions  exist  ("yes"  or  "no").  In  the 
examples  above,  state  "no"  for  all  three  functions.  However, 
if  B  and  C  are  mutually  exclusive,  then  "yes"  should  be 
indicated  in  the  column  headed  "Alternate  Step(s)"  for  both  B 
and  C.  (See  relation  No.  4  in  Figure  ni-2  on  page  B-16.  ) 

d.  Under  "Activity  Description"  name  the  Function  in  a  single 
word  verb,  or  a  brief  verb  phrase,  indicating  the  general 
operation;  e.g.,  "detect,"  or  "determine  distance  to  target,  " 

e.  Specify  the  output  state  of  each  function  in  relaticn.  to  its  input 
state;  i.  e.  ,  describe  altered  conditions  from  prior  specified 
(and  implied)  conditions. 

f.  Under  "Imposed  Requirement"  enter  the  reference  number  of 
the  related  system  effectiveness  requirement  from  the  list 
prepared  in  Step  1.  c.  ,  if  indicated. 


Example 

The  following  hypothetical  example  illustrates  the  technique  in  skelct.H 
form  in  order  to  be  brief.  A  more  complete  example  would  give  oetailed 
u  ta,  but  would  require  more  space  than  is  justified  to  demonstrait-  the  use 
of  the  procedures. 
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System:  Bench  checkout  of  electronic  item  X  (checkout  procedures 

available) 

a.  Requirements 

1)  [The  list  of  criteria  to  be  met  within  specified  tolerances] 
(In  a  realistic  example,  those  criteria  and  tolerances 
would  actually  be  stated.  ' 

2)  Time  limitation  for  entire  checkout  of  X  =  T  minutes. 

3)  Probability  required  for  successful  checkout  =  P. 

4)  X  18  ready  for  installation. 

b.  Input  state  conditions: 

1)  [Checkout  equipment]  are  available.  (These  should  be 
itemized.  ) 

2)  X  is  on  bench. 

3)  Technician  has  checkout  procedure  manual  open  to  first 
page  of  instructions, 

4)  [Required  power]  is  available  at  the  bench  (specify;  e,  g. , 
lOV.D.C.;  115  VAC,  60  cps). 

c.  [Intermediate  characteristics]  (In  checkout  procedures,  these 

are  generally  specified  by  the  output  criteria.  4s  listed  under 

"Requirements''  above. ) 

d.  Coiistraints; 

1)  Room  temperature  is  between  yj^^C  and  y^^C  (for  optimal 
performance  by  technician), 

2)  Sufficient  time  (T  -0, +t)  is  available  for  complete  check¬ 
out;  technician  is  not  likely  to  be  interrupted  while 
checking  out  X. 

3)  Checkout  table  is  not  made  of  any  material  which  conducts 
electrical  current. 

4''  Technician  stands  on  rubber  mat  during  checkout. 

e.  Establish  functions; 
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Graphic  Model  Data  Form 


System 

Function  Bench  Checkout  of  X 


PEF 

Unit 

No, 

Prior 

Necessary 

Step(s) 

Alter¬ 
nate 
Step(s ) 

Activity  Description 

Output  State 

Imposed 

Require¬ 

ment 

A 

Input 

Set  up  equipment 

(Each  piece  of 
equipment  listed 
with  its  required 
location) 

B 

A 

No 

Supply  power  to 
equipment 

(List  all  indica¬ 
tions  that  equip¬ 
ment  is  ON) 

C 

B 

No 

Checkout  X 

(List  of  measure-  1 
ment  data  speci-  ' 
fied  in  3.  a.  1 ) 

2)  L  3)) 

D 

C 

No 

Secure  equipment 

X  is  ready  for  | 

installation 

J 

C.  Procedural  Steps  at  the  Element  Level 


One  may  proceed  directly  to  the  elemental  level  of  system  description, 
or  one  may  describe  PEF  Units  at  a  level  of  specificity  intermediate  between 
lunctions  and  elemeiits.  For  large  complex  systems,  intermediate  levels 
may  need  to  be  generated  in  order  to  be  able  to  manage  further  analysis 
accurately  down  to  the  elemental  level.  For  example,  "Bench  Checkout  of 
X"  may  be  considered  a  subfunction  of  "Checkout"  of  a  very  large  system 
which  involves  design,  construction  and  checkout  of  many  pieces  of  equip¬ 
ment. 
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The  notation  which  is  used  to  designate  levels  of  specificity  is 
illustrated  in  Figure  II- 2  below.  Note  that  the  first  level  refers  to  Functions 
and  the  last  level  to  elements.  A  system  may  be  analyzed  to  two  or  more 
levels,  and  different  numbers  of  levels  may  be  used  for  different  systems. 
However,  for  any  one  system,  the  same  number  of  levels  should  be  used 
throughout  the  analysis.  In  the  example  to  be  presented  later  in  this 
section,  only  two  levels  of  specificity  are  indicated;  B.3,  for  example, 
will  represent  the  third  element  in  Function  B. 


PEF  Unit  r“ 
Notation  : 

C 

1 

2 

5 

1 

Levels  of 
Specif’city 
■  ■  ■  I 

1 

2 

i 

1 

1 

I  3 

) 

J  

1 

i 

4 

- r 

Example  ' 

i 

j 

Checkout 

Bench 
Checkout 
of  X 

H - 

1  Supply 

1  Power  to 
i  Equipment 

t 

f 

Place  ON- 
OFF  Switch 
on  X  in  ON 
Position  1 

Figure  II-2.  The  four  levels  of  specificity  of 
a  four- digit  PEF  Unit  (C.  1.  2.  5) 


The  Steps 

1.  At  the  top  of  the  GMDF,  identify  the  Function  to  be  analyzed  (with 
its  name  and  code  letter). 

2.  Referring  to  the  operational  procedures,  assign  a  number  to  each 
procedural  step.  (For  Function  A,  the  first  step  would  be,  A.  J, 
the  second  A.  2,  etc.  ) 

3.  Under  "Prior  Necessary  Step(s)"  list  the  code  number  of  the 
previous  step(s)  that  must  have  been  accomplished  before  this 
step  can  possibly  be  performed,  according  to  the  operational 
procedures. 

4.  Under  "Alternate  Step(s)"  indicate  whether  or  not  an  alternate 
approach  may  be  taken  to  arrive  at  the  output  state. 
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5. 


Describe  the  man-machine  operation  clearly  and  completely  under 
"Activity  Description.  " 

6.  Under  "Output  State"  describe  all  measurably  altered  features  of 
the  system  resulting  from  the  prior  activity. 


Example 

Referring  to  the  example  begun  under  Section  B.  3.  ,  we  will  look  at 
Function  B,  "Supply  Power  to  Equipment,  " 

Graphic  Model  Data  Form 


Function _ B.  Supply  Power  to  Equipment 


PEF 

Prior 

Alter- 

Imposed 

Unit 

Necessary 

nate 

Activity  Description 

Output  State 

Require- 

No. 

Step(s) 

5tep(s) 

ment 

B.  1 

(Last  step 
of  F unc¬ 
tion  A) 

No^ 

Place  AC  switches  to  power 
supplies  PS|  and  PS^  to  ON 
position.  (Observe  that  AC 

PSj  and  PS^ 
are  operative 

ON  lamps  are  lighted  on 

PSj  and  PS^.  ) 

B.  2 

It  II 

No 

Place  ON-OFF  switches  to 

Gp  a.nd  G-^ 

equipment  Gj,  G2  ^nd  G^ 

are  operative 

to  ON  position.  (Observe 
that  ON  lamps  are  lighted 
on  G],  G2  and  G3.  ) 

B.  3 

ir  ft 

No 

Press  ON  button  on  equip¬ 
ment  G3.  (Observe  ON 
lamp  is  lighted  on  G^.  ) 

G5  is  operative 

B.4 

B.  1 

No 

Three  minutes  after  B.  1, 

PSj  and  PS2 

place  DC  switches  to 
power  supplies  PS  j  and 

PS^  to  ON  position.  (Ob¬ 
serve  that  DC  ON  lamps 
are  lighted  on  PSj  andPS2.  ) 

are  operative 

B.  5 

B.  2 

No 

Place  ON-OFF  switch  on  X 

X  is  operative 

B.  3 

in  ON  position.  (Observe 

B.4 

that  ON  lamp  is  lighted  onX.) 

While  alternate  approaches  may  be  possibio,  if  the  procedures  do  not 
indicate  that  f.act,  none  are  listed. 
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III.,  CONSTRUCTING  THE  GRAPHIC  STATE  SEQUENCE  MODE.L  (GSSM) 


A.  jLiitroduction 


The  GSSM  is  a  diagrammatic  representation  of  the  system.  It  rpecifies 
not  only  what  the  system  is  expected  to  do,  but  also  what  the  system  is 
capable  of  doing.  As  a  result,  the  constructor  must  be  thoroughly  familiar 
with  the  system  and  all  documents  associated  with  it. 


The  GSSM  comprises  three  primary  components  to  symboli;,:e  the 
system  processes  and  their  interrelations:  (1)  "boxes"  (or  rectangles) 
represent  operations,  i.e.,  PEF  Units,  at  any  level  of  specificity; 

(2)  lines  connect  the  rectangles  and  represent  instantaneous  states  of  the 
system;  and  (3)  connecting  symbols  are  used  to  display  relations  among 
states.  Definitions  of  the  connecting  symbols  appear  in  Figure  III-l. 


AND  symbol  requires  the  simultaneous 
existence  of  two  or  more  conditions  for 
the  subsequent  action  to  occur;  State  S  = 
State  P  plus  State  Q  plus  State  R. 


P  or  P 


EXCLUSIVE  OR  symbol  specifies  that  only 
one  of  two  or  more  input  or  output  states 
can  exist  at  any  one  time;  State  S  =  State  P 
or  State  Q  £r  State  R. 

DOT  symbol  indicates  that  a  single  con¬ 
dition  has  two  or  .more  simultaneous  effects; 
State  S  =  State  P  =  State  Q  5  State  R  and  X 
and  Y  and  Z  occur, 

CROSSED  DOT  indicates  that  a  single  con¬ 
dition  has  two  or  more  possible  effects, 
but  specifies  that  only  one  of  the  alternative 
subsequent  actions  can  occur;  State  S  =  State  P 
P  =  State  Q  =  State  R  but  either  X  or  Y  or  Z 
can  occur. 


Figure  ni-1.  Definitions  of  connecting  symbols. 

Figure  III-2  demonstrates  some  basics  of  GSSM  construction  utilizing 
the  three  components. 
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Procedures  for  Constructing  the  GSl?M 


1,  Begin  by  representing  the  input  state  of  the  system  by  a  line 
ing  to  one  or  more  rectangles. 

2,  Using  JTigure  III-3  as  a  general  model  of  any  single  PEF  Unit, 
proceed  to  draw  the  diagram,  approaching  each  PEF  Unit  with 
the  following  series  of  questions  and  using  the  GMDF  data  as  a 
guideline;  the  answer  to  each  question  must  be  *‘yes,  "  for  the 
diagram  to  be  complete  anl  accurate: 


y 

z 

(input  state)'^ 

X 

1 

(output  state)'" 

Figure  HI- 3.  A  PEF  Unit 

a.  Is  it  necessary  to  the  system  that  x  occurs? 

b.  Is  y  completely  adequate  for  x  to  occur?  (H  additional  input 
conditions  are  needed,  those  must  be  included. ) 

c.  Is  y  just  sufficient?  That  is,  does  the  occurrence  of  x  require 
the  existence  of  the  entire  state  described  by  y? 

d.  Is  y  worded  so  as  to  describe  the  state  ftilly?  Are  all  relevant 
features  included  as  observable  or  measurable  conditioi.s  of 
the  system? 

e.  Does  the  diagram  illustrate  ail  occurrences  to  which  y  may 
give  rise? 

f.  If  X  is  not  the  only  operation  that  might  lie  between  y  and  z, 
for  this  particular  system,  are  the  alternatives  also  inr’  ..led? 

g.  Are  alternative  operations  meaningful?  (If  alternative  activitie 
are  possible  -  and  they  almost  always  are  -  determine  which 
ones  might  reasonably  be  expected  to  occur  sometimes,  and 
which  ones  ars  only  observed  possibilities  so  as  to  be  likely 
never  to  occur  at  all.) 

h.  Does  z  fully  express  the  total  change  in  the  system  resulting 
from  X  and  only  x? 


i.  Is  z  complete;,  that  is-  does  it  specify  the  alternative  oetcornes 
of  X? 

j.  Is  z  worded  so  as  tr  represent  an  observable,  measurable 
condition  of  the  system? 

3.  Illustrations  of  the  use  of  the  questions.  Figure  III-4  shows  how 
the  questions  in  Section  III.  3.2.  lead  to  correct  GSSM  con-itruction. 

4.  Eyampie  at  the  Function  Level 

a.  Referring  to  the  sample  GMDF  of  II.  B.  2.  e,  the  G^SSM  would 
initially  appear  as  shown  in  Figure  III- 5,  with  system  states 
clearly  defined. 

It  is  apparent  that  the  functions  (not  the  states)  could  apply 
to  the  checkout  of  any  system  involving  electrically  powered 
equipment. 

b.  Assume  that  some  of  the  checkout  of  X  must  occur  before 
power  is  supplied  to  the  equipment;  some  of  the  checkout 
can  occur  before  or  after  power  is  supplied  and  some  mc&t 
occur  after  power  is  supplied.  In  that  event,  Functions  E 
and  C  are  not  clearly  separable  as  indicated  in  Figure  III-5. 

At  the  function  level,  however.  Figure  III-5  is  still  acceptable 
because  it  is  primarily  descriptive  and  satisfies  the  definition 
of  a  function  (see  Section  L  B. );  a  more  specific  diagram 
would  no  longer  be  system  independent. 

A  second  breakdown  may  be  needed  to  incorporate  the  new 
information.  Figure  III-6  illustrates  how  that  would  be  done. 

5.  An  Example  at  the  Element  Level 

Referring  to  the  example  in  11.  C.  7. ,  the  GSSM  would  be  constructed 
as  shown  in  Figure  III-7.  In  order  to  conserve  space,  note  that 
the  code  numbers  in  the  boxes  of  the  diagram  refer  to  the  ’’Activity 
Descrij>tions”  in  the  example  of  11.  C,  7. 

The  features  to  be  observed  in  the  example  are  these: 

a.  B.  1,  B.  2,  cind  B.  3  may  be  performed  in  any  order  sequentially, 
or  simultaneously  by  different  persons. 
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examples  If  application 


INCOHRECT 


a.  A  PEr  Uju’c  which 
m>'»t  b«  travers«d 
in  ayitcm  opera* 
tion  Mt'ST  BE 
NECESSAKY  (o 
•yeton  pi^rformaitce 


Swltcli  El  1. 

Obaerve  that 

Eouipment  X  „ 

in  ON  poaltlon 

lighted 

ia  operative 

Equipment  X  cannot  be  operative  unleaa  SI  ia  ON,  but 
it  ran  be  operative  If  lamp  la  not  lighted  or  if  operator 
doea  not  tee  that  Ll  ^a  lighted. 


b.  '^hc  inptit  atate 
rr.uat  ba  cfitn* 
pletely  adequate 


Information 
rega  rdlna 
fdatuni  ijla 
recei e«d 


Blank  paper 

ia  on  deak  ^  X 

Recorder 

writea  in* 

Paper  contalna 
the  following 
data: 

Pencil  in  ^  ^ 

formation 

1.  ^ 

hand  la  polaed 

on  paper 

2. 

over  paper 

3. 

lnfor<Mation 
rega  rdtng 
fdata  i  and  3] 
la  received 


Pape:  and  pencil  ^ 

Recorder 
writer  in- 

Paper  rontaina 
the  following 
d.xia-  ^ 

are  in  hand  ^ 

! 

formation 

1  on  paper 

1. 

One  cannot  write  information  without  having  lecelvrd  it. 
The  PLF  Unit  deacription  ttaelf  ealla  for  paper,  pencil 
(or  pen)  and  the  data  to  be  written  do-jm. 


c.  The  input  atate 
muat  not  be  ex* 
ceaeive  foi-  the 
occurrence 
apeclficd  by  the 

PEr  Uni; 


Information 
rag arding 
(datum  I  r la 


Illank  paper 
ia  on  dealt 


Pencil  in 
hand  la  poiaed 
ovar  paper 


t 


Recorder 
writea  In* 
formation 
on  paper 


Information 
regarding 
(data  2  and  3j 
ia  received 


Paper  rontalna 
the  follo«idnf 
data:  . 

1, '  '  ~ 

2,  _ 

3.  _ 


I31anh  paper 
la  on  deak 


Pencil  in 
hand  ia  polaed 
over  paper 


Recorder 
writer  infor* 
matlon  on  paper 


Information 
regarding 
(  Jata  1,2.  3.4 
and  S)  ia 
received 


d.  The  input  etatc 
mua,  be  worded 
•  o  aa  fully  and 
objectively  to 
deacrlb*  the 
aiate 


Typed  tnforma* 
tion  regardin 
I. _  ■ 

in  the  trana* 
m:tter'a  handa 


Typed  tnforma- 
tion  regarding  ( 

_!■ _ .  I  _ J’ 


1 T  ranamit 

3.  [clc.  1  . 

Typed  data 
a  re  in  the  ^ 

Trannmit 

1  Data 

are  recorded  by''  tranamltter'a 

Data 

receiver 

hand  a 

Data  are 
recorded 
by  receiver 


e.  The  diagram  muat 
■  how  ■!«  occurren* 
res  to  which  the 
input  elate  givea 
rlae 


Typed  Informa¬ 
tion  rrgarding 

3.  fete, 
in  Iranamltler'a 
hand* 


1.  .2.  i 

Send 

T  ranamit 

—  fr.FT 

copiea 

Dita 

are  recorded 

by  receiver 

Coplea  of 
data  en* 
route  to 


ret  elver 
(enumera¬ 
ted) 


The  copiea  cannot  be  aent  until  they  are  madr.  and  the 
tranamltter  need  not  wait  until  he  tranamita  the  data 
before  making  the  Xerox  copiea  and  aendlng  them  off. 


f  tgurr  n*'4.  CiSSM  roii«tru< f ten  priitriplea. 


fiontinijcd  nexi  page) 


I'  I  4 


igure  111-6.  Example  of  a  possible  second  level  G5SM 
(from  Figure  III-5), 


b.  The  syptem  requires  only  that  the  equiprr»«^rit  be  '.  perative,  it 
does  not  require  that  .lamps  light.  Indicator  lamps  serve  to 
increase  the  probability  that  an  operative  state  will  be  reached, 
but  they  are  not  essential  to  system  performance. 

c.  Power  supplies  generally  will  operate  if  the  DC  voltage  Is 
turned  on  sooner  than  the  manufacturer  recommends. 

However,  the  load  on  the  supply  may  be  excessive,  resulting 
in  a  reduction  in  life  a-id  reliability. 

d.  Note  that  the  comments  regarding  the  three  minute  wait  would 
tiot  be  necessary  if  a  safety  timing  mechanism  made  it 
impossible  to  switch  DC  ON  before  three  minutes  had  elapsed. 
This  illustrate.^  the  need  to  know  the  system  in  detail. 

e.  If  is  assumed  that,  for  this  particular  system,  X  cannot  be 
turned  ON  unless  B.  1,  B.  2,  and  3.  3  are  performed  first. 
However,  investigation  may  prove  that  X  may  be  turned  on 
at  any  time;  if  that  were  trie,  the  input  state  for  B.  5  would 
be  identical  to  the  input  state  for  B.  1 .  B.  2  and  B.  5, 

6.  Example  Revised 

Acsume  that  the  GSSM  constructor  cannot  understand  why  B.  5 
must  occur  after  B.  1,  B.  2,  and  B,  3;  so  he  consults  one  of  the 
engineers  and  discovers  that  B.  5  can  only  occur  after  the  power 
supplies  are  turned  on  and  after  G.  5  is  turned  on,  but  not 
necessarily  after  G.  1,  G.  2  and  G.  3  are  on.  The  GSSM  for 
Function  B  would  have  to  be  correlated  to  look  like  tiia  structure 
shown  in  Figure  III-8-  (Note  that  the.  verbiage  is  omitted  in 
order  to  conserve  space.) 
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C.  Construction.  Rules  for  Special  Kinds  of  Situatio!:s 

1.  Alternative  activities  or  activity  sequences  should  begin  at  the 
common  input  state  and  end  at  the  common  output  state,  even  if 
elements  must  be  repeated. 

For  example,  assume  the  function  "going  home  from  the  airport"; 


X  is  just  outside 
the  airport 


Going  home 
from  the 
airport 


X  is  at 
home 


> 


X  may  take  a  bus  (B)  or  hail  a  taxi  (T).  If  X  takes  a  bus,  he  may 
walk  home  from  the  bus  stop  (W)  or  take  a  taxi  (Tg).  If  X  chooses 
to  cake  a  taxi  from  the  bus  ctop,  he  may  be  able  to  find  one  locally 
available  (F)  or  he  may  need  to  call  for  one  via  the  corner  tele¬ 
phone  (C).,  * 


The  correct  procedure 
is  to  follow  the  above 
rule  to  start  and  end  at 
the  same  common  states 


Do  not  try  to  minimize  the 
number  of  blocks  in  the  diagram 
(even  though  it  presents  the 
same  diagrammatic  information 
as  the  correct  version) 


By  using  the  correct  procedure,  one  can  analyze  each  total 
strategy  independently. 

Note:  Other  alternatives  are  conceivable,  but  based  upon  one's  knowledge 
of  the  system,  they  are  not  meaningful.  For  an  absurd  example,  X  might 
conceivably  find  a  pair  of  skates  and  skate  home.  For  a  more  sane  example, 
X  could  have  driven  ?  car  to  the  bus  stop  and  left  it  there  until  his  return. 
However,  based  on  our  knowledge  of  X  (or  lack  of  parking  spaces  near  the 
the  bus  stop)  the  probability  is  almost  zero  that  he  would  use  that  strategy; 
perhaps  X  has  never  owned  a  car,  he  is  too  young  to  drive,  or  his  wife 
always  uses  the  car. 
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Where  alternative  activities  are  net  exclusive,  establish  diagram¬ 
matic  combinations  which  force  them  to  be  exclusive. 

For  example,  assume  that  X  told  his  wife  (C.^)  that  he  might  call 
her  when  he  arrived  at  the  airport  and  that  he  might  either  call  a 
taxi  (C,p)  to  take  him  home  or  ask  her  to  pick  him  up,  depending  on 
his  feelings  at  the  moment.  Thus,  X  may  call  for  a  taxi  or  call 
his  wife  or  both. 


The  correct  procedure  is 
to  establish  exclusive 
alternatives 


Do  not  try  to  use  an 
and/or  symbol 


This  enables  independent  ana¬ 
lysis  of  the  three  possibilities 

Where  alternative  activities  depend  upon  other  contingencies,  those 
contingencies  are  summated  with  the  existing  state  of  the  system. 
For  example,  in  the  example  under  1  above,  assiune  that  if  a  taxi 
is  available  (T’)  when  X  gets  off  the  bus,  he  will  take  it,  otherv/ise 
he  will  walk. 


T' 


It 

4.  Always  avoid  redundant  paths  . 

Example  1: 

Assume  an  operator  has  thiee  checkout  switch)' s  to  ope-^arv. 

SI  turns  on  one  light,  Li,  and  S2,  tu;^tis  on  Subsequently,  33 
turns  off  L2  and  leaves  LI  on. 


(conditions  for 
checkout) 


Checkout 


LI  is  on  ^ 
is  off 


The  correct  diagram  gives 
all  necessary  information 


Do  not  indicate  a  continuation  that  is  already  specified  by  the 
diagram 


LI  is  on 
L2  is  off 


Example  2: 

If  an  activity  is  required  (e.  g.  ,  for  checkout)  and  it  is  essential 
that  the  activity  does  not  change  the  state  of  the  system,  do  not 
«dd  additional  paths;  tlie  state  description  indicates  the  require^-^e.it. 


correct 


Wrong 


State  ^ 

State 

■  J 
7 

1 

y 

i 


otate 


State 


The  example  which  was  carried  through  the  previous  three  rules  cannot 
readily  be  used  to  illustrate  this  rule.  Thus,  it  is  to  be  noted  that  eveiy 
rule  may  not  oe  applicable  to  all  systems;  every  system  must  be  ap- 
preached  uniquely. 
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IV. 


CONSTRUCTING  THE  MAl  HZMATrCAJL 
SEQUENCE  MOEEL  (MSSM) 


A  T^tT' 

^  4  J  \  X 


A.  Intro ‘Auction 


The  MSSM  is  one  or  a  set  c-f  equations  which  express  the  relation 
between  the  probability  of  achieving  the  system  output  state  and  the  proba¬ 
bilities  of  accomplishing  the  PE'F  Units  which  the  system  comprises.  The 
information  for  the  mathematical  model  comes  directly  from  the  completed 
GSSMi  so  it  is  assumed  that  the  graphic  model  is  complete  and  accurate 
before  the  steps  ir  this  section  are  taken. 

Constructing  the  MS5M  is  relatively  easy  once  the  basic  rules  are 
understood.  At  this  time,  it  is  uncertain  v/hether  that  simplicity  reflects 
the  present  youthful  stage  of  development  of  the  technique  or  is  an  inherent 
characteristic  of  the  technique;  the  latter  seems  more  likely. 

Step-by-step  instructions  for  constructing  the  mathematical  model 
will  be  presented.  An  example  vwdli  illustrate  each  step;  the  symbols 
shown  in  Figure  III-l  will  be  the  keys  to  establishing  the  correct  relatr.ons 
among  tlie  components  of  the  model.  In  addition,  some  matheniatical 
symbols  defined  in  Figure  IV- 1  will  also  be  used. 


B.  Basic  Principles 

All  Mathcr!?atical  Stare  Sequence  .Mode3s  are  expressions  equating  the 
probability  of  an  outpar.  state  to  the  detcrminiirg  probabiliHes.  The  .fvrnda- 
mental  principles  of  MSSM  constrcctioii  depend  upon  connectir^-  symbols, 
as  follows: 

Rule  1:  Thv  crebability  of  the  output  stite  of  a  requli*ed  set  cf  PEF 

Units  is  rcTaved  to  the  product  of  the  pTv''babilities  associated 
with  those  PEi'  Units. 

The  first  rule  applies  to: 

a.  A  series  ox  any  numhior  oi  PEF  U«its  with  no  connecting 
symbols  betv/ecn  them  (Sev  No.  1  in  Figure  2II-2),  and 

b.  Any  arrajigement  of  PEF  Units  connected  only  by  DOT 
and  AND  symbois;  (See  Nos.  2  and  5  in  Figure  111-2). 

The  mathematical  expression  for  all  such  arrangements  is 


Symbol 

meaning 

Example 

P(  ) 

The  probability  of 
AAfhatever  is  contained 
in  the  parentheses. 

1 

P(B.  3)  =  the  probability  of  the  required 
output  state  of  element  3  of  Funcrion  B.. 

(It  is  assumed  that  the  input  state  to 

B.  3  initiated  that  elemental  personnel/ 
equipment  function.  } 

n 

n 

i=i! 

1 

1 

i 

The  product  of  the  n 
values  following  the 
symbol. 

Assuming  the  system  has  four  Functions, 
as  shown  in  Figure  111-5, 

4  D 

n  P(i)  =  n  P{i)  =  P(A)P(B)P{C)PCD) 
i=l  i=A 

That  is,  the  symbol  says  "multiply  the 
four  probabilities,  "  substituting  for  i 
all  letters  from  A  to  D. 

m 

2 

j=i 

The  sum  of  the  m 
values  following  the 
symbol 

1 

1 

?  X  =  X,  +  X,  +  X,  +  -f 

pj  j  1  2  3  4  5 

Just  as  for  the  product  symbol,  the 
summation  symbol  says,  "substihate 
for  j  all  whole  numbers  from  1  to  m.  " 

•  *  •  i 

Three  dots  mean 
"and  so  on" 

n 

AxBxCx...xn  is  the  same  as  11  i  ; 

i=A 

often,  it  is  simply  written  A  x  B  x  C  x  . . . 

rr* 

A  +  B  +  C  +  ...+m  is  the  same  as  2  j  ; 

j=A 

often,  this  is  simply  written  asA  +  B  +  C... 

Figure  iV-i,  r^athernatical  symbois  which  are  used  in  MSSM. 
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n 


P(output)  =  P(in)  n  P(i) 

i=l 

where  P{in)  is  the  probability  of  the  input  state,  and  i  represents  the  code 
numbers  assigned  to  the  component  PEF  Units. 

Thus,  for  No.  1  in  Figure  111-2,  the  MSSM  is  P(z)  =  P(y)P{A)P(B). 
For  Nos.  2  and  3  in  Figure  III- 2,  the  models  are  identical, 

P(z)  =  P(y)P(A)P(B)P{C). 

Referring  now  to  Figures  III-7  and  III-8,  if  no  alternate  procedures 
were  possible  (i,  e. ,  if  all  diamonds  and  crossed  dots  were  eliminated) 
the  MSSM  for  both  GSSM  constructions  would  be  the  same: 

5 

P(x  is  operative)  =  P{output)  =  P{Ao)  H  P(B.i)  = 

i=l 

P(Aq);?(B.  1)P(B.  2)P(B.  3)P<B.4)P(B.  S) 
where  P(Aq)  is  the  probability  of  the  output  state  oi  Function  A. 


Rule  2;  The  probaoility  of  the  output  state  of  an  alternative  set  of 

PEF  Units  is  related  to  the  weighted  sum  of  the  probabilities 
associated  with  those  PEF  Units. 

The  second  rule  applies  to  alternative  approach  possibilities,  where 
the  input  state  for  two  or  more  PEF  Units  stems  from  a  CROSSED  DOT  (see 
No.  4  in  Figure  111-2).  The  general  mathematical  expression  for  that 
arrangement  is: 

m 

P{output)  =  P(in)  2  K.P(j) 

j=l  ^ 

where  K.  is  a  constant  associated  with  P{j),  and  j  represents  one  of  the 
altematilres. 

Thus,  for  No.  4  of  Figure  III- 2,  the  MSSM  has  two  parts: 

I.  P(a)  =  P(y)  [Kaj  P(Aj)  +  Ka2P(A2)  +  Ka3P(A3)] 

II.  P(z)  =  P(a)P(B)  according  to  the  first  rule. 
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To  obtain  the  overall  M5SM,  the  two  parts  are  combined  by  substitution: 


3 

P(a)  =  P(B)P(y)  [  2  Ka-  P(Ai)l 

j=l  •* 


m 


It  may  be  noted  that  it  is  always  true  that 
tive  approaches. 


K.  =  1.  0  for  any  set  of  alterna- 


Rule  3:  The  sum  of  the  probabilities  of  alternative  output  states  of  a 
PEF  Unit  is  directly  related  to  the  total  probability  of  the 
output  state  for  that  PEF  Unit. 

The  third  rule  applies  to  alternative  output  states  and  generally 
occurs  less  frequently  than  the  first  two.  The  general  mathematical 
expression  for  that  arrangement  is: 

m  m 

2  P(j)xP{in)  2  p(L) 

j=l  j=l 


where  L  is  the  PEF  Unit  whose  ou^ut  is  split  by  an  EXCLUSIVE  OR  connec 
ting  symbol. 

For  example*  assume  that  a  computer  must  be  fed  a  complex  problem 
for  which  three  different  unpredictable  answers  (or  range  of  values)  are 
possible;  this  rule  applies  if  each  possible  answer  must  be  followed  by  a 
different  procedure  each  of  which  leads  ultimately  to  the  system  output 
state.  This  situation  is  aaagrammed  in  Figure  IV- 2. 


Results  indicate  r  ^ 
Results  indicate  s 


Results  indicate  t 


Figure  IV- 2,  Hypothetical  example  of  an  OR  ouqjui 


The  MSSM  for  Figure  IV- 2  is: 

P(r)  +  P(s)  +  P(t)  =:  P(y)  (P(Ly)  +  PCLg)  +  P(Lt)] 

where,  for  example,  P(Lj.)  is  the  probability  that  output  state  r  will  follow 
after  L  occurs. 
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c. 


Procedural  Steps 


It  will  be  necessary  to  write  equations  for  portions  of  the  GSSM  ac  a 
time.  Therefore,  to  aid  construction  of  the  MSSIvl,  it  will  be  helpful  to 
assign  reference  symbols  to  certain  states  of  the  system.  To  do  this,  it 
may  be  helpful  to  construct  a  skeletal  GSSM  as  illustrated  in  Figure  IV-3. 

Step  1 

Using  lower  case  letters,  assign  a  different  reference  letter  at  the 
following  locations  in  the  GSSM: 

a.  Just  before  a  CROSSED  DOT, 

b.  Just  after  a  diamond,  i.  e.  ,  EXCLUSIVE  OR.  (This  overlaps 
with  the  next  two  steps.  ) 

c.  If  a  diamond  is  associated  with  a  previous  CROSSED  DOT, 
assign  the  same  letter,  but  with  different  numerical  subscripts, 
to  all  lines  leading  into  the  diamond,  and  assign  that  same 
letter  v/ithout  a  subscript  to  the  line  going  out  from  the  diamond. 
(See  output  of  Function  A  in  Figure  IV-3.  ) 

d.  If  a  diamond  appears  at  the  output  of  a  PEF  Unit,  assign  the 
same  letter,  but  wdth  different  numerical  subscripts  to  all 
lines  leading  away  from  the  diamond,  and  assign  the  same 
letter  without  a  subscript  to  the  line  going  into  the  diamond. 

(See  elements  B.  3,  B.  4,  and  B.  5  in  Figure  IV-3.  ) 

Step  Z 

Starting  at  the  beginning  of  the  GSSM,  apply  the  appropriate  rule 
between  every  sequential  pair  of  reference  letters.  If  the  reference  letters 
have  subscripts,  observe  whether  Rule  2  or  Rule  3  is  to  be  applied,  then 
proceed  as  follows: 

a.  If  Rule  2  applies,  write  equations  relating  each  subscript 
reference  letter  to  the  preceding  reference  letter,  (i.  e.  ,  to 
the  state  of  the  system  just  before  the  CROSSED  DOT).  Then 
write  the  symbolic  form  of  the  following  equation:  Probability 
of  the  reference  letter  having  no  subscript  equals  the  sum  c'' 
the  probabilities  of  each  of  the  same  reference  letters  with 
subscripts. 

b.  If  Rule  3  applies,  write  equations  relating  each  subscripted 
reference  letter  to  the  next  subsequent  reference  letter(s). 
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1.  P(r)  =  P(in)P(A.  1)  =  P{A.  1)  See  Note  No.  1 

2.  P{si)  =  K|P{r)P(A.2)  See  Note  No.  2 

3.  P(82)  =  K2P(r)P(A.  3)P(A.4)P{A.  5)  =  K2P(r)  P(A.  i) 

4.  P^sj)  =  K3P(r)P(A.6)P(A.7) 

5.  P(s)  =  P(si)  +  P(S2)  ■'  P(s3) 

=  P(r)  [KjP(A.2)  +  K,p(A.3)P(A.4)P(A.  5)  +  K3P{A.  6)P{A.  7)  ] 

6.  P{t)  =  P(s)P(B.  1)P(B.2)P(B.  3) 

7.  P(t)  =  P(ti)  +  P(t2)  See  Note  No,  3 


8.  P(n)  =  P(tj)P(B.4)  +  P(t2)P(3.  5) 

9.  P{out)  =  P(u)P{C.  1)P(C,2)P(C.  3)P(C.4)P{C.  5)P(C.  6)  =  P{u)  n  P(C.  1) 

i  =  l 

Subatituting  then,  and  assuming  that  PCtj)  =  ^(*2^  =1/2  P{t), 

6 

P{out)  =  1/2  P(t)  [P(B.4)  +  P(B.5)]  P{c.n 

6  3 

J’<out)  =  1/2  P{s)  [P(B.4)  +  P(B.  5)1  n  P(C.i)][  n  P{B.1)1 

i=l  i=l 

3  5  6  3 

P(out)  =  1/2  P(A.  1)  [  r  P(s,)]  f  2:  P(B.j)][n  P(c.i)][  n  P{B.i)] 

j  =  l  •'  j=4  i  =  l  i=I 

3 


where  21  P{s.) 
j  =  l  J 


=  KiP{A.2)  h  K^PCA.  3)P(A.4)P(A.  5)  +  K3P(A.  6)P{A.  7) 
Figure  IV-3.  A  hypothetical,  skeletal  GSSM. 
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Also  write  the  symbolic  form  of  the  following  equation: 
Probability  of  the  reference  letter  having  no  subscript  equals 
the  sum  of  the  probabili*ics  of  each  of  the  same  refe-ence; 
letters  with  subscripts. 


Step  3 

After  the  entire  series  of  equations  are  written  for  the  system, 
start  with  the  last  equation  and  perform  the  following  operation  repeatedly 
until  no  reference  letters  appear  in  the  equation  relating  P(in)  to  P(out), 
always  keeping  P(out)  on  the  left  side  of  the  equation: 

Substitute  for  P(refcrence  letter)  its  equivalent  value,  as  determined 
in  a  prior  equation.  For  e.xample,  if 

P(a)  =  P(in)P(A) 

and  P(out)  =  P(a)  [Ki  P(B>  +  K2P{C)] 
then  P(out)  =  P(in)P(A)  [KiP(B)  +  K2P(C)] 

D.  Example  and  Notes 

The  equations  accompanying  Figure  IV-3  illustrate  the-application  of 
the  steps  in  the  preceding  section.  Some  clarifying  comments  will  be  made 
in  the  form  of  notes  to  which  the  equations  make  reference. 

Note  1 :  It  is  always  assumed  that  the  probability  of  the  input 
state  to  the  total  system  is  1.0. 

Note  Z\  The  K  values  are  proportio.nality  constants,  indicating 
the  relative  extent  to  which  the  associated  alternative  is  used 
in  comparison  with  the  other  possible  alternative(s).  K  values 
arT^’ither  determined  by  observation  or  they  are  assigned 
arbitrarily  for  analytical  convenience  (i.  e.  ,  one  might  wish  to 
investigate  the  difference  it  would  make  to  the  system  if 
K|  =1.0  and  K2  =  =  0,  rather  than  if  K2  =  1.0  and  Kj  = 

K3  =0,  or  if  K3  =  1.  r.  and  Kj  =  -  0). 

Note  3:  Experience  or  knowledge  of  the  system  should  be 
adequate  to  estimate  the  relative  occurrences  of  alternative 
outcomes  of  a  per sonnel/equipment  function.  If  no  estimation 
is  po.ssiblc,  assume  equal  likelihoods  of  obtaining  all  possible 
outcome.s.  Therefore,  in  the  example,  if  no  guess  is  better 
th.an  any  other,  as  sumeP{tj)  =  P(t2)  =  1/2  P(t). 


V.  ALLOCATING  PROBABILITIES 


A.  Introduction 

This  section  is  concerned  with  the  procedure  for-  allocating  the  proba¬ 
bility  components  of  the  PEF  Unit  standards.  Thuse  are  the  probabilities 
with  which  the  required  output  states  --  shown  on  the  GSSM  --  must  be 
accomplished  in  order  for  the  overall  system  effectiveness  requirements 
to  be  met. 

In  order  to  allocate  probabilities  of  accomplishing  PEF  Units,  one 
must  first  determine  a  means  for  specifying  the  relations  among  the  PEF 
Units  so  as  to  be  able  to  assign  a  weighting  or  index  value  to  each  unit 
according  to  its  relative  expectancy  of  accomplishm.ent.  To  date,  no  means 
exists  for  deriving  a  reliable  and  meaningful  index  which  reflects  that 
expectancy.  However,  in  an  effort  to  test  the  technique,  the  "data  store" 
developed  by  Payne  and  Altmaii  was  used  to  derive  a  probability  index.  ^ 

The  figures  in  the  "data  store"  were  treated  as  if  they  were  weighting 
values,  rather  than  as  actual  probabilities  of  performance. 

B.  General  Procedure 


To  describe  the  procedure,  it  wi.il  be  assumed  that  an  Index  of  Task 
Accomplishment  (IOTA)  is  derivable  for  each  PEF  Unit.  If  we  let  P(  \  = 
the  probability  of  accomplishing  the  PEF  Unit  in  the  parentheses  and  I^{  )  = 
the  IOTA  for  the  same  PEF  Unit,  then,  in  general, 

P(  )  =  )]  (V-1) 


That  is,  the  probability  of  accomplishr.icnt  is  a  definable  function  of  the 
index  of  accomplishment. 

Step  1 

Referring  to  the  system  effectiveness  requirements,  substitute  in  the 
MSSM  the  required  value  for  P(out). 

Step  2 

As  a  result  of  that  simple  substitution,  all  probabilities  in  the  MSSM 
become  required  probabilities  of  accomplishment,  i.  e.  ,  P(  )  values. 

^  Payne,  D.  ,  and  Altman,  J.  W.  An  Index  of  Electronic  Equipment  Opera¬ 
bility,  Pittsburgh,  Pennsylvania:  American  Institute  for  Research, 
January  J  962. 
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P{out)  =  f‘rp(PEF  Units)] 


/nr  *%\ 

\y  -c} 


where  V  is  »  -’ifferent  function  from  f  in  Equation  V-1.  Since 
Equation  V-1  is  true,  then 

P(out)  =  f'  [f  {la(PEF  Units)}]  {V-3) 


Therefore,  substitute  for  each  P(  )  in  Equation  V-2  its  equivalent, 
respective  ffl^C  )]• 

Step  3 

Solve  Equation  V-3  for  the  values  of  IOTA. 

Step  4 

Substitute  the  values  into  Equation  V-1  one  at  a  time  and  compute 
each  P{  ). 


C.  Concluding  Example 

An  exceedingly  simple  example  will  be  used  to  illustrate  the  approach 
and  conclude  the  discussion.  Assume  a  system  comprises  two  PEF  Units 
In  series,  so  that  the  MSSM  is 

B 

P(out)  =f'[P{  )]  =  n  P(i)  =  P(A)P(B)  (V-4) 

i=A 

From  the  system  effectiveness  requirement,  we  find  that  P(out)  =  0.  95. 
Thus, 


P(out)  =  0.  95  =  P(A)P(B) 


(V-5) 


Assume  that  the  ’’data  store"  was  used  to  obtain  the  following  IOTA 
estimates: 


Ij^CA)  =  0.  87  and  Ij^CB)  =  0.  92 


Also,  it  has  been  hypothesized  that: 
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(V-6) 


P  =  f(I^)  =k(l-y  +  V' 


where  k  is  constant  for  any  given  system.  Substituting  in  Equation  V-5, 
0.  95  =  [k{l-Ia(A)}  +  I^(A)]  [k{l-I^(B)}  + 

0.  95  [k{  1  -0.  87}  +  0.  87]  [k{  1-0.  92}  +  0.  92] 

0.  95  =  (0.  1  3  k  +  0.  87)  (0.  08  k  +  0.  92) 

95  -  1. 04  k2  +  18.  92  k  +  80.  04 
k^  +  1 8.  2  k  -  1 4.  4  =  0 


18.  2  a:  Vl8.  2^  +  57.  6 


=  -9.1  ±  9.  86  =  0.  76;  -18.96 


From  Equation  V-6 

P(A)  =  k(l-  I^)  +  la  =  0.  76(0.  13)  +  0.  87 

P(A)  =  0.  969 

P(B)  =  0.  76(0.  08)  +  0.  92 

P(B)  =  0.  981 

Thus,  in  order  to  meet  the  system  effectiveness  requirement  that 
P(out)  =  0.  95,  the  required  probabilities  of  accomplishing  the  PEF  Units 
must  be  P(A)  =  0.  969  and  P(B)  =  0.  981. 


Note  that  Equation  V-6  requires  0  <  1. 0 
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